Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.070580615s FAIL (2025-09-29 07:30:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.070617735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.340701127s FAIL (2025-09-29 07:29:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.340732929s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.110661694s FAIL (2025-09-29 07:28:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.110700007s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.641826161s FAIL (2025-09-29 07:27:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.641929808s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.132483226s FAIL (2025-09-29 07:26:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.132518833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.083295925s FAIL (2025-09-29 07:25:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.083320883s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.934742203s FAIL (2025-09-29 07:24:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 5.934775828s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.185767279s FAIL (2025-09-29 07:24:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.185793367s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.304550913s FAIL (2025-09-29 07:22:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.304725144s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.121348093s FAIL (2025-09-29 07:21:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.12137285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.170133316s FAIL (2025-09-29 07:20:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.170191347s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.178868731s FAIL (2025-09-29 07:19:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.178896986s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.229818542s FAIL (2025-09-29 07:18:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.229847726s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.461775304s FAIL (2025-09-29 07:17:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.461806102s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.275044868s FAIL (2025-09-29 07:17:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.275075427s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.158516057s FAIL (2025-09-29 07:16:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.158551785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.139269964s FAIL (2025-09-29 07:15:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.139304058s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.214255765s FAIL (2025-09-29 07:15:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.214291433s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.109263908s FAIL (2025-09-29 07:14:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.109298956s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.327673459s FAIL (2025-09-29 07:13:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.327710109s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.210555285s FAIL (2025-09-29 07:13:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.210578649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.401167606s FAIL (2025-09-29 07:12:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.401205357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.246009598s FAIL (2025-09-29 07:11:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.246097406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.167826571s FAIL (2025-09-29 07:10:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.167860797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.300681561s FAIL (2025-09-29 07:09:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.300716536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.208258805s FAIL (2025-09-29 07:08:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.20829333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.119184257s FAIL (2025-09-29 07:07:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.119213895s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.152303686s FAIL (2025-09-29 07:06:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.152339063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.115498089s FAIL (2025-09-29 07:05:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.115527536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.075840966s FAIL (2025-09-29 07:04:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.075878498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.156862476s FAIL (2025-09-29 07:03:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.156895247s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.144635754s FAIL (2025-09-29 07:02:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.144668867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.448376387s FAIL (2025-09-29 07:02:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.448412024s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.288879991s FAIL (2025-09-29 07:01:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.288904917s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.528802523s FAIL (2025-09-29 07:00:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.52883734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.090530748s FAIL (2025-09-29 06:59:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.090561375s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.977952448s FAIL (2025-09-29 06:58:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 5.977986783s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.274430243s FAIL (2025-09-29 06:57:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.274461142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.035777987s FAIL (2025-09-29 06:56:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.035811312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.051666059s FAIL (2025-09-29 06:56:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.051700826s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.026579211s FAIL (2025-09-29 06:54:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.02661477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.004125684s FAIL (2025-09-29 06:54:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.004163346s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.030520287s FAIL (2025-09-29 06:53:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.030565264s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.090309889s FAIL (2025-09-29 06:52:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.090358772s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.254499092s FAIL (2025-09-29 06:52:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.254540501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.933303788s FAIL (2025-09-29 06:51:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 5.93333591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.001022932s FAIL (2025-09-29 06:50:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.001054222s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.042074967s FAIL (2025-09-29 06:50:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.042108572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.138821833s FAIL (2025-09-29 06:49:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.138859835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.195270452s FAIL (2025-09-29 06:48:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 29 06:41:00.181194479 (137h9m45.643286952s) [upload=120722]: Starting upload Sep 29 06:41:00.181297906 (137h9m45.643390379s) [upload=120722]: Done starting upload Sep 29 06:41:00.181482096 (137h9m45.643574569s) [upload=120722]: Uploading segments... Sep 29 06:41:00.181556399 (137h9m45.643648892s) [upload=120722]: Got next segment. Inline: false Sep 29 06:41:00.181601985 (137h9m45.643694468s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 06:41:00.181607776 (137h9m45.643700259s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 06:41:00.432663241 (137h9m45.894755714s) [seg_pos={0 0}, upload=120722]: Begin upload segment... Sep 29 06:41:00.432728205 (137h9m45.894820678s) [seg_pos={0 0}, upload=120722]: Done begin upload segment. Sep 29 06:41:00.432754124 (137h9m45.894846598s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.432861900 (137h9m45.894954383s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.433368625 (137h9m45.895461108s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.436558298 (137h9m45.898650781s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.437470897 (137h9m45.89956338s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438368234 (137h9m45.900460717s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.438847187 (137h9m45.90093966s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.439976147 (137h9m45.90206863s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.440125170 (137h9m45.902217654s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.441438081 (137h9m45.903530564s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.443691411 (137h9m45.905783884s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.445119691 (137h9m45.907212174s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.446594028 (137h9m45.908686512s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.448493746 (137h9m45.910586229s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.449140788 (137h9m45.911233271s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:00.450874088 (137h9m45.912966572s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Uploading piece... Sep 29 06:41:02.111479968 (137h9m47.573572451s) [upload=120722]: Next returned nil segment Sep 29 06:41:02.111535274 (137h9m47.573627757s) [upload=120722]: Waiting for error group managing segments... Sep 29 06:41:02.505279655 (137h9m47.967372148s) [noise=true , address=145.79.227.18:4005 , piece_id=ZLH3JC6QO7NMEIUWIPV7DFL456AY66MWUEQ7WJXCNF6XGSAOAA6Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507060486 (137h9m47.969152969s) [noise=true , address=145.79.227.11:4731 , piece_id=E55SCM4VEWXJQQ4PCMS522P453JPMT564XGHPWXQGIU2YQCRT7VQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.507741463 (137h9m47.969833946s) [noise=true , address=145.79.227.6:4257 , piece_id=RWK2HJDPU47LWMKZSA436ZI4T2UVGHNGM2FH66346TDL2UIES5IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.516901164 (137h9m47.978993647s) [noise=true , address=145.79.227.4:4186 , piece_id=2CAZDPXNZ2O4P4SSK53XPMZSP4MVBZS7IQX7ZHZEMO2I66F5OYGA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.517526335 (137h9m47.979618818s) [noise=true , address=145.79.227.19:4079 , piece_id=WO3OQM5QNZDC4OCO4RXXE2BEOSNN43XM3AJK7L2H4PS7QIWBOOBA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.548263313 (137h9m48.010355806s) [noise=true , address=45.91.116.12:4227 , piece_id=7YYYQSEPPCSJWUYBIMJUDEI2FP2M6J3T2UZTFBPMYJWZKWHVN4UA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.554649243 (137h9m48.016741736s) [noise=true , address=45.91.116.11:4117 , piece_id=RLF3UC45C2KZPM2VUUCMOIYXVFKRX656NZHAYCSH2G6QJKL23VDA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.575602401 (137h9m48.037694884s) [noise=true , address=45.91.116.13:4154 , piece_id=QYO63B33KZ7CMJOUZ4YPZFGGPFISQW5DQON4O3PPPIBARN2D7C2A, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.594535884 (137h9m48.056628367s) [noise=true , address=45.91.116.4:4939 , piece_id=X6KEO4UPCYW3HWXK6F6J7PDUWBAL5DEE77RAYMNH2PXIQKOLL76Q, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.597991635 (137h9m48.060084128s) [noise=true , address=77.90.31.19:4842 , piece_id=7P33B534BW5ODFADWJAGSJENBF6USQBEXJ3IRF2QZGNRKW5SGRIA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.605536753 (137h9m48.067629236s) [noise=true , address=45.91.116.11:4119 , piece_id=N7R6XVPVKPYWSTPPVN33OS2AUH6X4KAZIPEML566SELWWPKDRKHA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.606701620 (137h9m48.068794103s) [noise=true , address=77.90.31.15:4699 , piece_id=KZ4ZWE423BBZ2JEC7KURVTJWOQMCIKRQAPF3VT2SAMRKARBFM5OA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618151523 (137h9m48.080244016s) [noise=true , address=45.91.116.6:4974 , piece_id=DWJOU65HBHOVIWQ6UZ6JWTU2OXKNSYR2UPF3FNY4CU5G2S2SU3EQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: <nil> Sep 29 06:41:02.618182532 (137h9m48.080275015s) [seg_pos={0 0}, upload=120722]: Segment reached optimal threshold of 13 pieces. Sep 29 06:41:02.618346805 (137h9m48.080439288s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618359068 (137h9m48.080451551s) [noise=true , address=77.90.31.11:4554 , piece_id=NYKGYO7Z53BTV6NBYAZ7FLQHYZLXVK4BUJ6SXYZGJVUBUB2A65IQ, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618428571 (137h9m48.080521074s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618441706 (137h9m48.080534189s) [noise=true , address=77.90.31.14:4661 , piece_id=NBN4TMHIMVLITA67MYNGBB2FKMYZFTPOU4MIKZT2ROV7ISX4TNLA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618475199 (137h9m48.080567682s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 29 06:41:02.618483836 (137h9m48.080576319s) [noise=true , address=45.91.116.10:4081 , piece_id=FESUGLRMWMOXK7I35DNR6SACW5XTC4VAKWX4W7H6DFCPXG7MLPTA, seg_pos={0 0}, upload=120722]: Long tail cancellation due to optimal threshold Sep 29 06:41:02.618509605 (137h9m48.080602088s) [seg_pos={0 0}, upload=120722]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 29 06:41:02.618619545 (137h9m48.080712028s) [upload=120722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 29 06:41:02.618629133 (137h9m48.080721616s) [upload=120722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 29 06:41:02.942512436 (137h9m48.404604919s) [upload=120722]: Done uploading segments... Sep 29 06:41:02.942536411 (137h9m48.404628894s) [upload=120722]: Upload finished. err: <nil> Duration: 6.195397674s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)