Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.175802438s FAIL (2025-10-04 09:31:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.175864015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.883023575s FAIL (2025-10-04 09:31:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.883062921s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.072017658s FAIL (2025-10-04 09:30:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.072050832s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.955947433s FAIL (2025-10-04 09:29:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.955987891s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.666688094s FAIL (2025-10-04 09:28:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.66672231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 6.133787261s FAIL (2025-10-04 09:28:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.133815075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 6.030780005s FAIL (2025-10-04 09:27:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.030817867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.89967807s FAIL (2025-10-04 09:26:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.899701043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.02971779s FAIL (2025-10-04 09:25:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.029759581s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 9.419405205s FAIL (2025-10-04 09:25:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 9.419465018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.767446922s FAIL (2025-10-04 09:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.767478853s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.813294858s FAIL (2025-10-04 09:23:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.813326127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 6.072889194s FAIL (2025-10-04 09:22:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.07291975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.921778923s FAIL (2025-10-04 09:21:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.921811203s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 6.009832775s FAIL (2025-10-04 09:20:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.009862641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 6.315375061s FAIL (2025-10-04 09:19:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.315408246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 6.166100831s FAIL (2025-10-04 09:18:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.166164283s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 6.00582204s FAIL (2025-10-04 09:18:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.005881493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 6.275854698s FAIL (2025-10-04 09:17:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.275878843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 6.088314344s FAIL (2025-10-04 09:16:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.088340744s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 6.086055356s FAIL (2025-10-04 09:15:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.086089931s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 6.076586759s FAIL (2025-10-04 09:14:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.076628227s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 6.077076024s FAIL (2025-10-04 09:13:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.07711035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 6.021074475s FAIL (2025-10-04 09:13:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.021103631s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 6.337380154s FAIL (2025-10-04 09:12:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.337413298s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.155253961s FAIL (2025-10-04 09:11:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.155288848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 6.126037408s FAIL (2025-10-04 09:10:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.126070882s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.050299344s FAIL (2025-10-04 09:09:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.050330964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.100367716s FAIL (2025-10-04 09:08:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.100400077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 5.932162088s FAIL (2025-10-04 09:07:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.932195441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 5.852966703s FAIL (2025-10-04 09:07:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.852990417s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 6.107618467s FAIL (2025-10-04 09:06:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.107657862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 6.38103165s FAIL (2025-10-04 09:05:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.381070383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 6.013760927s FAIL (2025-10-04 09:03:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.013792536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 6.117566979s FAIL (2025-10-04 09:03:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.117599212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.847011313s FAIL (2025-10-04 09:03:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.847075165s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.883403591s FAIL (2025-10-04 09:02:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.883438027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 6.127687808s FAIL (2025-10-04 09:01:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.127722063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 6.037395961s FAIL (2025-10-04 09:01:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.037429475s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 6.567152271s FAIL (2025-10-04 09:01:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.567329779s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 6.121642931s FAIL (2025-10-04 09:00:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.121675505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 5.905301615s FAIL (2025-10-04 08:59:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.905338957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 5.67135817s FAIL (2025-10-04 08:58:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.671393387s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 5.859591911s FAIL (2025-10-04 08:57:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.859650963s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 5.835930414s FAIL (2025-10-04 08:57:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.83596574s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 5.779889535s FAIL (2025-10-04 08:56:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.779997682s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 6.107719827s FAIL (2025-10-04 08:55:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.107753553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 6.230770062s FAIL (2025-10-04 08:54:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.230807463s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 6.118568578s FAIL (2025-10-04 08:54:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.118607092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 6.68670936s FAIL (2025-10-04 08:53:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.686829399s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)