Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.270142565s FAIL (2025-10-26 12:17:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.270171341s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.968692731s FAIL (2025-10-26 12:16:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 5.968728377s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.96939694s FAIL (2025-10-26 12:16:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 5.96944404s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.922290711s FAIL (2025-10-26 12:15:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 5.922323022s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.464744434s FAIL (2025-10-26 12:14:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.464778459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.059010485s FAIL (2025-10-26 12:13:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.059040382s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.119305353s FAIL (2025-10-26 12:12:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.119339056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.267057648s FAIL (2025-10-26 12:11:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.267096331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.174618338s FAIL (2025-10-26 12:10:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.174650249s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.184005767s FAIL (2025-10-26 12:09:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.184044812s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.035869668s FAIL (2025-10-26 12:08:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.03590159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.173945858s FAIL (2025-10-26 12:08:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.173980364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.160457632s FAIL (2025-10-26 12:07:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.160491327s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.213587321s FAIL (2025-10-26 12:07:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 6.213620253s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002931468s FAIL (2025-10-26 12:07:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 12:06:58.742481533 (790h35m44.204574017s) [upload=695451]: Starting upload Oct 26 12:06:58.742556716 (790h35m44.20464921s) [upload=695451]: Done starting upload Oct 26 12:06:58.742643773 (790h35m44.204736266s) [upload=695451]: Uploading segments... Oct 26 12:06:58.742750606 (790h35m44.204843089s) [upload=695451]: Got next segment. Inline: false Oct 26 12:06:58.742787566 (790h35m44.204880039s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:06:58.742792656 (790h35m44.204885129s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:06:59.004481091 (790h35m44.466573585s) [seg_pos={0 0}, upload=695451]: Begin upload segment... Oct 26 12:06:59.004583135 (790h35m44.466675618s) [seg_pos={0 0}, upload=695451]: Done begin upload segment. Oct 26 12:06:59.004647177 (790h35m44.46673966s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.004838933 (790h35m44.466931416s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005231891 (790h35m44.467324374s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.005627924 (790h35m44.467720397s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006063204 (790h35m44.468155687s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006228689 (790h35m44.468321162s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006407219 (790h35m44.468499692s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006633579 (790h35m44.468726052s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.006715907 (790h35m44.46880838s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007099016 (790h35m44.469191499s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007161475 (790h35m44.469253948s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007385692 (790h35m44.469478165s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007647491 (790h35m44.469739974s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.007668741 (790h35m44.469761224s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.008253464 (790h35m44.470345947s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:06:59.009194536 (790h35m44.471287019s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Uploading piece... Oct 26 12:07:00.735130277 (790h35m46.19722276s) [upload=695451]: Next returned nil segment Oct 26 12:07:00.735153862 (790h35m46.197246346s) [upload=695451]: Waiting for error group managing segments... Oct 26 12:07:01.022839741 (790h35m46.484932234s) [noise=true , address=145.79.227.23:4546 , piece_id=DZRHPXBRQMTR6KDVLL2OTAIZ4L7QFZWK465FUKYA5A4HJETL6FAA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.025436185 (790h35m46.487528668s) [noise=true , address=145.79.227.16:4910 , piece_id=DPGBZBYOIP2JVAMNF2GJLSCJST3W75KG6B7YVFUIJ7LMN6HEMFRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.030832240 (790h35m46.492924713s) [noise=true , address=145.79.227.7:4294 , piece_id=UAHS3ABTO7RVKN2T42JDPFWXIK5CJSQCQH2O46H6KMFI4GWIU2FA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.034546813 (790h35m46.496639296s) [noise=true , address=145.79.227.2:4114 , piece_id=WAMHYOS2U7226AP4C6VMIBVX5SF4D6R5SSHWMGCPUFHT5J7IVFUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.041338867 (790h35m46.50343135s) [noise=true , address=145.79.227.4:4187 , piece_id=RVAYXHIFVEHHGSYTPUMXWEL3YI44PLJNQEGBVHA6LFAYRRYCSMLQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.049598554 (790h35m46.511691038s) [noise=true , address=145.79.227.5:4222 , piece_id=VIWOCZNW3WZ4KJBHWS22ELX7C4G23MAFXB6OGWOOPXAJHDCAGRRQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.051381159 (790h35m46.513473642s) [noise=true , address=145.79.227.24:4582 , piece_id=KU6K4R26HKX2SV3XOOJIHMAR7TPBTWHAF4T327HKPYHQ43AHBYPQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.057772609 (790h35m46.519865093s) [noise=true , address=145.79.227.4:4185 , piece_id=MAEGS34GTUX3KDJV5GGJOTX4D4QYHKE2AFU3CKTEIALOQNDMLWKA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.110222835 (790h35m46.572315318s) [noise=true , address=77.90.31.10:4519 , piece_id=EO3CKB5AAEJGCO6SCZ75SV4UG7TUM3MFZPQYXTE6SFU2OI2HLFSQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.157728885 (790h35m46.619821368s) [noise=true , address=145.79.227.15:4875 , piece_id=QIIAIUOYUMZDJGWYABABWU7TNYSKMEGFZISITVMNHXARTCPTGIMQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.225029639 (790h35m46.687122122s) [noise=true , address=77.90.31.13:4626 , piece_id=W363FHME6FCO4XLCCTNWYYRPWZG45T5JQUWXHCZJUAQH7T37JE4Q, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.364557957 (790h35m46.82665044s) [noise=true , address=77.90.31.17:4772 , piece_id=NTCHUUP4RHZTDPNPE56UH5HIVFPF67DFK5NYATVSM5DCIPLT46YQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412449933 (790h35m46.874542406s) [noise=true , address=77.90.31.21:4879 , piece_id=5GHTN2B4EHJKTAMX66EUBHZIYFSDC7ESFBA3XVWNNJY5KTR2UIUQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: <nil> Oct 26 12:07:01.412479689 (790h35m46.874572182s) [seg_pos={0 0}, upload=695451]: Segment reached optimal threshold of 13 pieces. Oct 26 12:07:01.412613834 (790h35m46.874706317s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412626689 (790h35m46.874719162s) [noise=true , address=145.79.227.17:4971 , piece_id=6OKP6V2JOJM2GUQVVODD7DAVNTTJW5AKNYUABTLLC4NSSTTHERZA, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412696562 (790h35m46.874789035s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412721159 (790h35m46.874813642s) [noise=true , address=145.79.227.21:4473 , piece_id=6OZ5TYEW4BCSFWUTVBKTRHNE2R3HRFJZEAHWGTHR7HL2GWT5VTNQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412730156 (790h35m46.874822639s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Oct 26 12:07:01.412742629 (790h35m46.874835113s) [noise=true , address=145.79.227.26:4654 , piece_id=DSN5OD34T7EZEFQZJSDT442JE5KXT337MN4YT622F47L377DE3OQ, seg_pos={0 0}, upload=695451]: Long tail cancellation due to optimal threshold Oct 26 12:07:01.412768289 (790h35m46.874860772s) [seg_pos={0 0}, upload=695451]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:07:01.412876254 (790h35m46.874968737s) [upload=695451]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:07:01.412886283 (790h35m46.874978766s) [upload=695451]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:07:01.735179381 (790h35m47.197271874s) [upload=695451]: Done uploading segments... Oct 26 12:07:01.735209559 (790h35m47.197302042s) [upload=695451]: Upload finished. err: <nil> Duration: 30.002958961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.89277979s FAIL (2025-10-26 10:21:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.892811159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.823375352s FAIL (2025-10-26 10:20:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.823413294s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.813566483s FAIL (2025-10-26 10:18:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.813595519s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.136431521s FAIL (2025-10-26 10:18:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.136463521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.202964047s FAIL (2025-10-26 10:17:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.202998672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.817048022s FAIL (2025-10-26 10:16:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.817080894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.922323078s FAIL (2025-10-26 10:16:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.922358225s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.072950401s FAIL (2025-10-26 10:15:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.073006438s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.937888718s FAIL (2025-10-26 10:15:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.937919316s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 11.884040077s FAIL (2025-10-26 10:15:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 11.884070947s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.807662207s FAIL (2025-10-26 10:14:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.807697964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.124671981s FAIL (2025-10-26 10:13:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.124707238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.096238809s FAIL (2025-10-26 10:12:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.096270077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 25.405457145s FAIL (2025-10-26 10:11:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 25.40549089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.049364783s FAIL (2025-10-26 10:09:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.049403287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.826299415s FAIL (2025-10-26 10:08:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.826327518s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.88840674s FAIL (2025-10-26 10:08:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.888435695s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.889423274s FAIL (2025-10-26 10:07:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.889451918s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 12.571661226s FAIL (2025-10-26 10:06:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 12.57169501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.050887671s FAIL (2025-10-26 10:06:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.050920944s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.070277986s FAIL (2025-10-26 10:05:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.07030615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.105936635s FAIL (2025-10-26 10:05:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.105990436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.901460102s FAIL (2025-10-26 10:04:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.901491903s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.879109048s FAIL (2025-10-26 10:03:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.879155677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.953926228s FAIL (2025-10-26 10:03:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.953960463s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.839776087s FAIL (2025-10-26 10:02:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.839807106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.034673682s FAIL (2025-10-26 10:01:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.034703679s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.908305783s FAIL (2025-10-26 10:01:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.908339128s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.003128999s FAIL (2025-10-26 10:00:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.003164146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.01426677s FAIL (2025-10-26 09:59:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.014294432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.914733863s FAIL (2025-10-26 09:58:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.914761576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.78192091s FAIL (2025-10-26 09:58:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.7819574s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.205325315s FAIL (2025-10-26 09:57:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 6.205395589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.914160453s FAIL (2025-10-26 09:56:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, Piece ID: PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ) context deadline exceeded; piecestore: (Node ID: 12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, Piece ID: 6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ) context deadline exceeded ======LOGS====== Oct 26 09:37:34.123071205 (788h6m19.585163678s) [upload=693243]: Starting upload Oct 26 09:37:34.123157710 (788h6m19.585250193s) [upload=693243]: Done starting upload Oct 26 09:37:34.123221812 (788h6m19.585314285s) [upload=693243]: Uploading segments... Oct 26 09:37:34.123433054 (788h6m19.585525527s) [upload=693243]: Got next segment. Inline: false Oct 26 09:37:34.123468111 (788h6m19.585560584s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 09:37:34.123473151 (788h6m19.585565624s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 09:37:34.371973433 (788h6m19.834065906s) [seg_pos={0 0}, upload=693243]: Begin upload segment... Oct 26 09:37:34.372032737 (788h6m19.83412521s) [seg_pos={0 0}, upload=693243]: Done begin upload segment. Oct 26 09:37:34.372057634 (788h6m19.834150107s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.372108300 (788h6m19.834200813s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373623074 (788h6m19.835715547s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.373860316 (788h6m19.835952829s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374152954 (788h6m19.836245427s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.374700806 (788h6m19.83679331s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375448481 (788h6m19.837540954s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.375998859 (788h6m19.838091362s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.377161182 (788h6m19.839253655s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378339846 (788h6m19.840432349s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.378778221 (788h6m19.840870704s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.379959250 (788h6m19.842051723s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.380473749 (788h6m19.842566232s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.381073312 (788h6m19.843165795s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.383435108 (788h6m19.845527581s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:34.385259823 (788h6m19.847352306s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Uploading piece... Oct 26 09:37:36.070745218 (788h6m21.532837731s) [upload=693243]: Next returned nil segment Oct 26 09:37:36.070766137 (788h6m21.53285862s) [upload=693243]: Waiting for error group managing segments... Oct 26 09:37:36.406762149 (788h6m21.868854632s) [noise=true , address=145.79.227.18:4005 , piece_id=QO5MPWZIVIBC2Z4KZXN5WCZT5K5EPWVVQANET4UTIMRSTRXRTI6A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.409600124 (788h6m21.871692597s) [noise=true , address=145.79.227.23:4545 , piece_id=CSR54JE45RBS764V42NL6US73UGKU47KFREREDNKZPCV2L2W47NQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.416939049 (788h6m21.879031522s) [noise=true , address=145.79.227.18:4006 , piece_id=OBAUIRAVT5TNOSXNWFIP7NX7LOAKCH5M7MMROPJHFX6YZATNARIQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.418279591 (788h6m21.880372074s) [noise=true , address=145.79.227.22:4511 , piece_id=TCOMYXXU7CFYAM73AUQ66IOF6XHAHFV6YKPGUEX4BQ3ZAAN7REKA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.426303711 (788h6m21.888396214s) [noise=true , address=145.79.227.6:4257 , piece_id=F3SHTZRLL2GHBRIGXTJZCMBIP2FQTE7CV7HI6BOTPGN66JAKMLEA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.430767821 (788h6m21.892860304s) [noise=true , address=145.79.227.21:4475 , piece_id=BQSABKMHRP3H67KYRBK2MKJTHHMCFRJRM23D2WTOUMJU2RMS5GNQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.431738389 (788h6m21.893830872s) [noise=true , address=145.79.227.25:4619 , piece_id=MFYNCKJTG5NSMCC4IQ6RAZIGRY6LYBJK4ZIC6N5HVVZFQTYXXSTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.439393947 (788h6m21.90148642s) [noise=true , address=145.79.227.17:4970 , piece_id=P6VHRKEVMDQAYGZIK5OOZK7IRHDGG24DD67BGMWKJGHDCPBMB2IA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.442868813 (788h6m21.904961297s) [noise=true , address=145.79.227.3:4151 , piece_id=DEQRVZTIXDG6OKYJRJP7EFRQ6KXOXR62T42WFVK6W6MJTTDN4NAQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.481214000 (788h6m21.943306473s) [noise=true , address=145.79.227.21:4474 , piece_id=NO7ZKII6YPPD3W4J75LG3MQ6JJDPXJSS77SHJKGSQ2QLQIXTPLTQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.485925793 (788h6m21.948018276s) [noise=true , address=145.79.227.3:4150 , piece_id=QZYKDSR3KBVVGTFHLD6POVZRX2GV6SQ2INT7D522NJZ5YGHMNCZA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.577169889 (788h6m22.039262362s) [noise=true , address=77.90.31.16:4733 , piece_id=PR6AHBMQ6ZES2HZCNCLOMCGVBVSNUB3RDZDNL3Z4HVB3REE7BTEQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616544336 (788h6m22.078636819s) [noise=true , address=77.90.31.18:4805 , piece_id=6PZQAUOOHEYBD554UTVBHKROER3L2OM2DXZT7TMR5FEGVVDD7AXQ, seg_pos={0 0}, upload=693243]: Done uploading piece. err: <nil> Oct 26 09:37:36.616576046 (788h6m22.078668519s) [seg_pos={0 0}, upload=693243]: Segment reached optimal threshold of 13 pieces. Oct 26 09:37:36.616738325 (788h6m22.078830798s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616745569 (788h6m22.078838052s) [noise=true , address=45.91.116.11:4117 , piece_id=VNBEA5VQKB4PRKPMPLB6TCMRQ4EIUICH27HCTP2GUI4NKZZFIVDA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616832614 (788h6m22.078925097s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616840730 (788h6m22.078933203s) [noise=true , address=45.91.116.4:4937 , piece_id=36AU7KAHTQXPWVVE4PGJGJJQD54V3I7D2JZX3RUDAG76MJSE6N7A, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616859936 (788h6m22.078952419s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Oct 26 09:37:36.616882380 (788h6m22.078974863s) [noise=true , address=77.90.31.14:4661 , piece_id=UAO2FLMNOVCG6WAELDJ33XUBXYDCMNOFFL5TXPEC7LY6UBA2SRYA, seg_pos={0 0}, upload=693243]: Long tail cancellation due to optimal threshold Oct 26 09:37:36.616922255 (788h6m22.079014738s) [seg_pos={0 0}, upload=693243]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 09:37:36.617562955 (788h6m22.079655538s) [upload=693243]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:37:36.617579246 (788h6m22.07967173s) [upload=693243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:37:36.914166186 (788h6m22.376258659s) [upload=693243]: Done uploading segments... Oct 26 09:37:36.914194310 (788h6m22.376286783s) [upload=693243]: Upload finished. err: <nil> Duration: 5.914192773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 18.679464128s FAIL (2025-10-26 09:56:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 11:51.222330643 (787h40m36.684423116s) [noise=true , address=77.90.31.12:4622 , piece_id=UL26HAE7NMBEOIBGHAD7MMIERZ6O3DP56YDDQU3GRTTCDEO5XIGQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.222787022 (787h40m36.684879505s) [noise=true , address=77.90.31.18:4823 , piece_id=YXHIIYK733X3ZG5L37ZFXXTLFZEKFTBN6375DK7S433WIQI63PZA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.223169571 (787h40m36.685262054s) [noise=true , address=45.91.116.12:4229 , piece_id=3U7JDSZN3BARD4KIVTEEABPQKCOISXJNHQUH76H7Z6YJDVXFE5VA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.223453251 (787h40m36.685545734s) [noise=true , address=145.79.227.11:4737 , piece_id=F6ANXEFIQ2V6IJFFNSRB56MHQZF5JCLQQBGCNZWQKGRDCF7SUKSQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.223785975 (787h40m36.685878458s) [noise=true , address=77.90.31.14:4670 , piece_id=OVXQ4YW4O34ZPKW235WJ5NMPMW5SCVNFDYXTUIFOIREYXLTFYMJA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.224102177 (787h40m36.68619466s) [noise=true , address=145.79.227.38:4002 , piece_id=2WN2ARMGOQLFHV24TBIOQWJHKV6FJZ6O6FBTASAEJKMJHW5EV3UQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.224574195 (787h40m36.686666678s) [noise=true , address=77.90.31.19:4863 , piece_id=L22445LDWTRC5SB7UONF5TKSNPH57XN4DIBH3SMXRSRIIKNZZIWA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.224872674 (787h40m36.686965157s) [noise=true , address=77.90.31.13:4635 , piece_id=KJBCXXOHTIHCQQACD26L52WFYLQZPMLTW3MN2QA7HM5R46DUSNYA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.225214946 (787h40m36.687307419s) [noise=true , address=45.91.116.14:4223 , piece_id=LXBPNQC6U666UP5G3QU5TDCULLERPFCOYMBR5ZOAOMKFW3IJZFKA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.225560494 (787h40m36.687652967s) [noise=true , address=77.90.31.15:4701 , piece_id=GRKLC6CH5BQNYTOIL6JDLO6DDYB26CLXLKMLV3EJLE2IRJO3PFEQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.225956848 (787h40m36.688049331s) [noise=true , address=77.90.31.16:4753 , piece_id=6HIZXMBAANFRBPEJYI6XMEO33QYG6KSCI3KCYNK37DBC6AW7G73A, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.226364384 (787h40m36.688456867s) [noise=true , address=145.79.227.7:4324 , piece_id=TOAE5LURPGZBD2OJUCMMW2WQSXFZPVOREP7HX2XD63667URT5SDA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.227523551 (787h40m36.689616034s) [noise=true , address=45.91.116.9:4280 , piece_id=4MKC26EJSOVQO6B63AIATIPSNKFSLEPOME6RSDEPM35INF2OFKKA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.227745984 (787h40m36.689838467s) [noise=true , address=77.90.31.21:4892 , piece_id=2UIEK666YKUL4TLMGCZNSRVMH2TZQNQZGILY4P3U6F6YJQBM26JQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.228765657 (787h40m36.69085813s) [noise=true , address=45.91.116.7:4044 , piece_id=YFWXOA5M6FKTZSDGRI6EZ7YBUT5X4WTBCXYI7HWOIT7AMLR5UNWA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.229181408 (787h40m36.691273882s) [noise=true , address=145.79.227.16:4931 , piece_id=VY2IF6TONVU7TBBWW5DBSEQMWV64FSTI54EDN6I7QCRHN3GHOUDA, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:51.230599488 (787h40m36.692691961s) [noise=true , address=145.79.227.5:4238 , piece_id=WG33KN3SBYNJLES3DTLVRKNJUFOYRVD6WI2SKUVVRXUKURVQX7CQ, seg_pos={0 0}, upload=692861]: Uploading piece... Oct 26 09:11:52.727018255 (787h40m38.189110738s) [upload=692861]: Next returned nil segment Oct 26 09:11:52.727082728 (787h40m38.189175201s) [upload=692861]: Waiting for error group managing segments... Oct 26 09:11:53.134330412 (787h40m38.596422885s) [noise=true , address=145.79.227.11:4737 , piece_id=F6ANXEFIQ2V6IJFFNSRB56MHQZF5JCLQQBGCNZWQKGRDCF7SUKSQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.141654859 (787h40m38.603747342s) [noise=true , address=145.79.227.5:4238 , piece_id=WG33KN3SBYNJLES3DTLVRKNJUFOYRVD6WI2SKUVVRXUKURVQX7CQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.145129877 (787h40m38.60722235s) [noise=true , address=145.79.227.7:4324 , piece_id=TOAE5LURPGZBD2OJUCMMW2WQSXFZPVOREP7HX2XD63667URT5SDA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.161148489 (787h40m38.623241012s) [noise=true , address=145.79.227.16:4931 , piece_id=VY2IF6TONVU7TBBWW5DBSEQMWV64FSTI54EDN6I7QCRHN3GHOUDA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.205126371 (787h40m38.667218854s) [noise=true , address=145.79.227.38:4002 , piece_id=2WN2ARMGOQLFHV24TBIOQWJHKV6FJZ6O6FBTASAEJKMJHW5EV3UQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.244490708 (787h40m38.706583181s) [noise=true , address=45.91.116.10:4092 , piece_id=7GVTGRP5BHVAOPGYVZXDORWLTPV37WJUQ5SV4Y7TFHC44YIBD6PA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.245945258 (787h40m38.708037731s) [noise=true , address=77.90.31.19:4863 , piece_id=L22445LDWTRC5SB7UONF5TKSNPH57XN4DIBH3SMXRSRIIKNZZIWA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.305496434 (787h40m38.767588927s) [noise=true , address=45.91.116.12:4229 , piece_id=3U7JDSZN3BARD4KIVTEEABPQKCOISXJNHQUH76H7Z6YJDVXFE5VA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.319540126 (787h40m38.781632599s) [noise=true , address=77.90.31.21:4892 , piece_id=2UIEK666YKUL4TLMGCZNSRVMH2TZQNQZGILY4P3U6F6YJQBM26JQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.329271826 (787h40m38.791364299s) [noise=true , address=45.91.116.14:4223 , piece_id=LXBPNQC6U666UP5G3QU5TDCULLERPFCOYMBR5ZOAOMKFW3IJZFKA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.332343686 (787h40m38.79443617s) [noise=true , address=45.91.116.9:4280 , piece_id=4MKC26EJSOVQO6B63AIATIPSNKFSLEPOME6RSDEPM35INF2OFKKA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.342035802 (787h40m38.804128276s) [noise=true , address=77.90.31.12:4622 , piece_id=UL26HAE7NMBEOIBGHAD7MMIERZ6O3DP56YDDQU3GRTTCDEO5XIGQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.373785539 (787h40m38.835878022s) [noise=true , address=77.90.31.18:4823 , piece_id=YXHIIYK733X3ZG5L37ZFXXTLFZEKFTBN6375DK7S433WIQI63PZA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.390469179 (787h40m38.852561662s) [noise=true , address=77.90.31.14:4670 , piece_id=OVXQ4YW4O34ZPKW235WJ5NMPMW5SCVNFDYXTUIFOIREYXLTFYMJA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: <nil> Oct 26 09:11:53.390500137 (787h40m38.85259261s) [seg_pos={0 0}, upload=692861]: Segment reached optimal threshold of 14 pieces. Oct 26 09:11:53.390630505 (787h40m38.852722988s) [noise=true , address=45.91.116.7:4044 , piece_id=YFWXOA5M6FKTZSDGRI6EZ7YBUT5X4WTBCXYI7HWOIT7AMLR5UNWA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b9DdnzjhU7zj2RSCba3ZWS7AZbZZ4YmDDF4QRq6dZC6YfskNz): protocol: expected piece hash; context canceled; EOF Oct 26 09:11:53.390639823 (787h40m38.852732296s) [noise=true , address=45.91.116.7:4044 , piece_id=YFWXOA5M6FKTZSDGRI6EZ7YBUT5X4WTBCXYI7HWOIT7AMLR5UNWA, seg_pos={0 0}, upload=692861]: Long tail cancellation due to optimal threshold Oct 26 09:11:53.390699396 (787h40m38.852791869s) [noise=true , address=77.90.31.13:4635 , piece_id=KJBCXXOHTIHCQQACD26L52WFYLQZPMLTW3MN2QA7HM5R46DUSNYA, seg_pos={0 0}, upload=692861]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp): protocol: expected piece hash; context canceled; EOF Oct 26 09:11:53.390705007 (787h40m38.85279748s) [noise=true , address=77.90.31.13:4635 , piece_id=KJBCXXOHTIHCQQACD26L52WFYLQZPMLTW3MN2QA7HM5R46DUSNYA, seg_pos={0 0}, upload=692861]: Long tail cancellation due to optimal threshold Oct 26 09:11:53.390724324 (787h40m38.852816797s) [noise=true , address=77.90.31.16:4753 , piece_id=6HIZXMBAANFRBPEJYI6XMEO33QYG6KSCI3KCYNK37DBC6AW7G73A, seg_pos={0 0}, upload=692861]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE): protocol: expected piece hash; context canceled; EOF Oct 26 09:11:53.390785290 (787h40m38.852877764s) [noise=true , address=77.90.31.16:4753 , piece_id=6HIZXMBAANFRBPEJYI6XMEO33QYG6KSCI3KCYNK37DBC6AW7G73A, seg_pos={0 0}, upload=692861]: Long tail cancellation due to optimal threshold Oct 26 09:11:53.390800329 (787h40m38.852892812s) [noise=true , address=77.90.31.15:4701 , piece_id=GRKLC6CH5BQNYTOIL6JDLO6DDYB26CLXLKMLV3EJLE2IRJO3PFEQ, seg_pos={0 0}, upload=692861]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Oct 26 09:11:53.390807993 (787h40m38.852900466s) [noise=true , address=77.90.31.15:4701 , piece_id=GRKLC6CH5BQNYTOIL6JDLO6DDYB26CLXLKMLV3EJLE2IRJO3PFEQ, seg_pos={0 0}, upload=692861]: Long tail cancellation due to optimal threshold Oct 26 09:11:53.390831508 (787h40m38.852923981s) [seg_pos={0 0}, upload=692861]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 26 09:11:53.390904788 (787h40m38.852997261s) [upload=692861]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 09:11:53.390909757 (787h40m38.85300224s) [upload=692861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 09:11:53.689354952 (787h40m39.151447445s) [upload=692861]: Done uploading segments... Oct 26 09:11:53.689380491 (787h40m39.151472974s) [upload=692861]: Upload finished. err: <nil> Duration: 18.679495569s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)