Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X..............X...........X..X..X..X....X......X.X.X.] duration: 24.206178103s FAIL (2025-10-09 14:37:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, Piece ID: VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A) context deadline exceeded ======LOGS====== Oct 9 14:36:40.651220117 (385h5m26.1133126s) [upload=338830]: Starting upload Oct 9 14:36:40.651349133 (385h5m26.113441616s) [upload=338830]: Done starting upload Oct 9 14:36:40.651536029 (385h5m26.113628522s) [upload=338830]: Uploading segments... Oct 9 14:36:40.651656769 (385h5m26.113749252s) [upload=338830]: Got next segment. Inline: false Oct 9 14:36:40.651703418 (385h5m26.113795901s) [upload=338830]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:36:40.651710802 (385h5m26.113803285s) [upload=338830]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:36:40.936651407 (385h5m26.39874389s) [seg_pos={0 0}, upload=338830]: Begin upload segment... Oct 9 14:36:40.936720228 (385h5m26.398812711s) [seg_pos={0 0}, upload=338830]: Done begin upload segment. Oct 9 14:36:40.936753933 (385h5m26.398846416s) [noise=true , address=77.90.31.17:4770 , piece_id=HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.936879482 (385h5m26.398971965s) [noise=true , address=45.91.116.5:4297 , piece_id=7DJ472Y6PLWP7OAMP4OXBNIMKSITZOJPXFVV3GRS4SXDM7XUA5IA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.937371860 (385h5m26.399464343s) [noise=true , address=45.91.116.14:4189 , piece_id=7QUQWAYO35I5THOIBMDSW2HXVCEADP2FMVIZBBBOCQS6FOKAN7YA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.937756011 (385h5m26.399848494s) [noise=true , address=145.79.227.2:4115 , piece_id=R5II227ZFNRRRDVSRCC6ESO3OORP5Z4TFSLXRJE2RAU4IIUB5P3A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938154048 (385h5m26.400246522s) [noise=true , address=45.91.116.5:4298 , piece_id=JPNFJ4HV7PHPX3OJG6A6FUDXXSPKP6ZF4DFPCUB7S4MJF2YNEWXQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938535294 (385h5m26.400627778s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938886683 (385h5m26.400979157s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.939415000 (385h5m26.401507483s) [noise=true , address=45.91.116.13:4153 , piece_id=5MQEGLW2FH32HZYAW2A2LF2HRZUZJQPXITORL5RW3VROEBJKTX3A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.939662191 (385h5m26.401754664s) [noise=true , address=145.79.227.14:4837 , piece_id=5UUJ4KX4TF5UX45UDVG4TI62ROJDWOVIO5PJUHD3XVN4NOZT3BEA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.940179366 (385h5m26.402271849s) [noise=true , address=145.79.227.5:4221 , piece_id=FWAR4HNBBRWFOPWYQYGXCKOPET6SX5AI7GMNWIEWCCYZ3SIUYJNQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.940908915 (385h5m26.403001398s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.941378439 (385h5m26.403470922s) [noise=true , address=45.91.116.6:4974 , piece_id=U434AHQ6AKZUIYS6QW5FNJVJG2ZPR5PGYLZVBDRA35HAKFY5L6YQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.941916414 (385h5m26.404008897s) [noise=true , address=45.91.116.4:4938 , piece_id=QPKQLFA27RATKOKJDNN6SZHFRTC2PL2SMO6S23NHM4OXXGL5Q6BA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942420834 (385h5m26.404513317s) [noise=true , address=145.79.227.7:4295 , piece_id=ACWNZ2FGU6TKPRS2ZWJ45QIADNG5X5BKHBHNMDLF5ROS3ZKAB6CQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942552865 (385h5m26.404645348s) [noise=true , address=145.79.227.18:4007 , piece_id=HQ5LR7Y33KCCKHNGFYPAX5GYAHXDFBOSJKYCV3SKR3VYLZKLNXKA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942895848 (385h5m26.404988332s) [noise=true , address=77.90.31.11:4555 , piece_id=VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:42.947248909 (385h5m28.409341392s) [upload=338830]: Next returned nil segment Oct 9 14:36:42.947272214 (385h5m28.409364697s) [upload=338830]: Waiting for error group managing segments... Oct 9 14:36:43.343292427 (385h5m28.8053849s) [noise=true , address=77.90.31.11:4555 , piece_id=VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.631743558 (385h5m29.093836031s) [noise=true , address=45.91.116.5:4298 , piece_id=JPNFJ4HV7PHPX3OJG6A6FUDXXSPKP6ZF4DFPCUB7S4MJF2YNEWXQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.634414945 (385h5m29.096507438s) [noise=true , address=45.91.116.4:4938 , piece_id=QPKQLFA27RATKOKJDNN6SZHFRTC2PL2SMO6S23NHM4OXXGL5Q6BA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.646431155 (385h5m29.108523638s) [noise=true , address=45.91.116.6:4974 , piece_id=U434AHQ6AKZUIYS6QW5FNJVJG2ZPR5PGYLZVBDRA35HAKFY5L6YQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.742193577 (385h5m29.20428606s) [noise=true , address=77.90.31.17:4770 , piece_id=HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.750669527 (385h5m29.21276201s) [noise=true , address=45.91.116.13:4153 , piece_id=5MQEGLW2FH32HZYAW2A2LF2HRZUZJQPXITORL5RW3VROEBJKTX3A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.768139724 (385h5m29.230232207s) [noise=true , address=145.79.227.14:4837 , piece_id=5UUJ4KX4TF5UX45UDVG4TI62ROJDWOVIO5PJUHD3XVN4NOZT3BEA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.769831225 (385h5m29.231923708s) [noise=true , address=145.79.227.5:4221 , piece_id=FWAR4HNBBRWFOPWYQYGXCKOPET6SX5AI7GMNWIEWCCYZ3SIUYJNQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.844486053 (385h5m29.306578526s) [noise=true , address=45.91.116.5:4297 , piece_id=7DJ472Y6PLWP7OAMP4OXBNIMKSITZOJPXFVV3GRS4SXDM7XUA5IA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.848304845 (385h5m29.310397328s) [noise=true , address=145.79.227.18:4007 , piece_id=HQ5LR7Y33KCCKHNGFYPAX5GYAHXDFBOSJKYCV3SKR3VYLZKLNXKA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.865687014 (385h5m29.327779497s) [noise=true , address=145.79.227.2:4115 , piece_id=R5II227ZFNRRRDVSRCC6ESO3OORP5Z4TFSLXRJE2RAU4IIUB5P3A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.893882193 (385h5m29.355974666s) [noise=true , address=145.79.227.7:4295 , piece_id=ACWNZ2FGU6TKPRS2ZWJ45QIADNG5X5BKHBHNMDLF5ROS3ZKAB6CQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.899482107 (385h5m29.36157459s) [noise=true , address=45.91.116.14:4189 , piece_id=7QUQWAYO35I5THOIBMDSW2HXVCEADP2FMVIZBBBOCQS6FOKAN7YA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.899534467 (385h5m29.36162695s) [seg_pos={0 0}, upload=338830]: Segment reached optimal threshold of 13 pieces. Oct 9 14:36:43.899687799 (385h5m29.361780282s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899696405 (385h5m29.361788888s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899754155 (385h5m29.361846638s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899759475 (385h5m29.361851948s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899774745 (385h5m29.361867228s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899787218 (385h5m29.361879701s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899810793 (385h5m29.361903266s) [seg_pos={0 0}, upload=338830]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:36:43.899904721 (385h5m29.361997204s) [upload=338830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:36:43.899913278 (385h5m29.362005761s) [upload=338830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:36:44.206048733 (385h5m29.668141226s) [upload=338830]: Done uploading segments... Oct 9 14:36:44.206074231 (385h5m29.668166714s) [upload=338830]: Upload finished. err: <nil> Duration: 24.206206657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X..............X...........X..X..X..X....X......X.X.X] duration: 30.003701495s FAIL (2025-10-09 14:37:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, Piece ID: VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A) context deadline exceeded ======LOGS====== Oct 9 14:36:40.651220117 (385h5m26.1133126s) [upload=338830]: Starting upload Oct 9 14:36:40.651349133 (385h5m26.113441616s) [upload=338830]: Done starting upload Oct 9 14:36:40.651536029 (385h5m26.113628522s) [upload=338830]: Uploading segments... Oct 9 14:36:40.651656769 (385h5m26.113749252s) [upload=338830]: Got next segment. Inline: false Oct 9 14:36:40.651703418 (385h5m26.113795901s) [upload=338830]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:36:40.651710802 (385h5m26.113803285s) [upload=338830]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:36:40.936651407 (385h5m26.39874389s) [seg_pos={0 0}, upload=338830]: Begin upload segment... Oct 9 14:36:40.936720228 (385h5m26.398812711s) [seg_pos={0 0}, upload=338830]: Done begin upload segment. Oct 9 14:36:40.936753933 (385h5m26.398846416s) [noise=true , address=77.90.31.17:4770 , piece_id=HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.936879482 (385h5m26.398971965s) [noise=true , address=45.91.116.5:4297 , piece_id=7DJ472Y6PLWP7OAMP4OXBNIMKSITZOJPXFVV3GRS4SXDM7XUA5IA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.937371860 (385h5m26.399464343s) [noise=true , address=45.91.116.14:4189 , piece_id=7QUQWAYO35I5THOIBMDSW2HXVCEADP2FMVIZBBBOCQS6FOKAN7YA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.937756011 (385h5m26.399848494s) [noise=true , address=145.79.227.2:4115 , piece_id=R5II227ZFNRRRDVSRCC6ESO3OORP5Z4TFSLXRJE2RAU4IIUB5P3A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938154048 (385h5m26.400246522s) [noise=true , address=45.91.116.5:4298 , piece_id=JPNFJ4HV7PHPX3OJG6A6FUDXXSPKP6ZF4DFPCUB7S4MJF2YNEWXQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938535294 (385h5m26.400627778s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.938886683 (385h5m26.400979157s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.939415000 (385h5m26.401507483s) [noise=true , address=45.91.116.13:4153 , piece_id=5MQEGLW2FH32HZYAW2A2LF2HRZUZJQPXITORL5RW3VROEBJKTX3A, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.939662191 (385h5m26.401754664s) [noise=true , address=145.79.227.14:4837 , piece_id=5UUJ4KX4TF5UX45UDVG4TI62ROJDWOVIO5PJUHD3XVN4NOZT3BEA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.940179366 (385h5m26.402271849s) [noise=true , address=145.79.227.5:4221 , piece_id=FWAR4HNBBRWFOPWYQYGXCKOPET6SX5AI7GMNWIEWCCYZ3SIUYJNQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.940908915 (385h5m26.403001398s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.941378439 (385h5m26.403470922s) [noise=true , address=45.91.116.6:4974 , piece_id=U434AHQ6AKZUIYS6QW5FNJVJG2ZPR5PGYLZVBDRA35HAKFY5L6YQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.941916414 (385h5m26.404008897s) [noise=true , address=45.91.116.4:4938 , piece_id=QPKQLFA27RATKOKJDNN6SZHFRTC2PL2SMO6S23NHM4OXXGL5Q6BA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942420834 (385h5m26.404513317s) [noise=true , address=145.79.227.7:4295 , piece_id=ACWNZ2FGU6TKPRS2ZWJ45QIADNG5X5BKHBHNMDLF5ROS3ZKAB6CQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942552865 (385h5m26.404645348s) [noise=true , address=145.79.227.18:4007 , piece_id=HQ5LR7Y33KCCKHNGFYPAX5GYAHXDFBOSJKYCV3SKR3VYLZKLNXKA, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:40.942895848 (385h5m26.404988332s) [noise=true , address=77.90.31.11:4555 , piece_id=VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ, seg_pos={0 0}, upload=338830]: Uploading piece... Oct 9 14:36:42.947248909 (385h5m28.409341392s) [upload=338830]: Next returned nil segment Oct 9 14:36:42.947272214 (385h5m28.409364697s) [upload=338830]: Waiting for error group managing segments... Oct 9 14:36:43.343292427 (385h5m28.8053849s) [noise=true , address=77.90.31.11:4555 , piece_id=VSEKLAOW52PPKCQAFEKWOA5MA2WCSMG7FDXI6II2AEQDBARUQLDQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.631743558 (385h5m29.093836031s) [noise=true , address=45.91.116.5:4298 , piece_id=JPNFJ4HV7PHPX3OJG6A6FUDXXSPKP6ZF4DFPCUB7S4MJF2YNEWXQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.634414945 (385h5m29.096507438s) [noise=true , address=45.91.116.4:4938 , piece_id=QPKQLFA27RATKOKJDNN6SZHFRTC2PL2SMO6S23NHM4OXXGL5Q6BA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.646431155 (385h5m29.108523638s) [noise=true , address=45.91.116.6:4974 , piece_id=U434AHQ6AKZUIYS6QW5FNJVJG2ZPR5PGYLZVBDRA35HAKFY5L6YQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.742193577 (385h5m29.20428606s) [noise=true , address=77.90.31.17:4770 , piece_id=HCA4GO5EDIQJ7PRQ3NLANTO7LC6PJKZ6OD4KTF2IJQFINJ4NH54A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.750669527 (385h5m29.21276201s) [noise=true , address=45.91.116.13:4153 , piece_id=5MQEGLW2FH32HZYAW2A2LF2HRZUZJQPXITORL5RW3VROEBJKTX3A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.768139724 (385h5m29.230232207s) [noise=true , address=145.79.227.14:4837 , piece_id=5UUJ4KX4TF5UX45UDVG4TI62ROJDWOVIO5PJUHD3XVN4NOZT3BEA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.769831225 (385h5m29.231923708s) [noise=true , address=145.79.227.5:4221 , piece_id=FWAR4HNBBRWFOPWYQYGXCKOPET6SX5AI7GMNWIEWCCYZ3SIUYJNQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.844486053 (385h5m29.306578526s) [noise=true , address=45.91.116.5:4297 , piece_id=7DJ472Y6PLWP7OAMP4OXBNIMKSITZOJPXFVV3GRS4SXDM7XUA5IA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.848304845 (385h5m29.310397328s) [noise=true , address=145.79.227.18:4007 , piece_id=HQ5LR7Y33KCCKHNGFYPAX5GYAHXDFBOSJKYCV3SKR3VYLZKLNXKA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.865687014 (385h5m29.327779497s) [noise=true , address=145.79.227.2:4115 , piece_id=R5II227ZFNRRRDVSRCC6ESO3OORP5Z4TFSLXRJE2RAU4IIUB5P3A, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.893882193 (385h5m29.355974666s) [noise=true , address=145.79.227.7:4295 , piece_id=ACWNZ2FGU6TKPRS2ZWJ45QIADNG5X5BKHBHNMDLF5ROS3ZKAB6CQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.899482107 (385h5m29.36157459s) [noise=true , address=45.91.116.14:4189 , piece_id=7QUQWAYO35I5THOIBMDSW2HXVCEADP2FMVIZBBBOCQS6FOKAN7YA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: <nil> Oct 9 14:36:43.899534467 (385h5m29.36162695s) [seg_pos={0 0}, upload=338830]: Segment reached optimal threshold of 13 pieces. Oct 9 14:36:43.899687799 (385h5m29.361780282s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899696405 (385h5m29.361788888s) [noise=true , address=145.79.227.6:4257 , piece_id=6FOSNA44D24CXTQOCFSDXLA4X46G43RXCUB7K2UQLDN5LAMKIDKA, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899754155 (385h5m29.361846638s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899759475 (385h5m29.361851948s) [noise=true , address=145.79.227.3:4151 , piece_id=3DP6RMIIBLFOOXH4FQRPTFYAQ54IYWQSO37LBZOYXW4YDA3ARLIA, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899774745 (385h5m29.361867228s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:36:43.899787218 (385h5m29.361879701s) [noise=true , address=145.79.227.16:4910 , piece_id=YQKXXJHUXEMGOXFSSDJ5ATHY4UKWQNSP2VLNTYDSSUNLHPO5ZYVQ, seg_pos={0 0}, upload=338830]: Long tail cancellation due to optimal threshold Oct 9 14:36:43.899810793 (385h5m29.361903266s) [seg_pos={0 0}, upload=338830]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:36:43.899904721 (385h5m29.361997204s) [upload=338830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:36:43.899913278 (385h5m29.362005761s) [upload=338830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:36:44.206048733 (385h5m29.668141226s) [upload=338830]: Done uploading segments... Oct 9 14:36:44.206074231 (385h5m29.668166714s) [upload=338830]: Upload finished. err: <nil> Duration: 30.003729228s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X.......X..............X...........X..X..X..X....X......X.X.] duration: 24.021424891s FAIL (2025-10-09 14:36:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: 5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A) context deadline exceeded ======LOGS====== Oct 9 14:35:34.557314854 (385h4m20.019407337s) [upload=338812]: Starting upload Oct 9 14:35:34.557409734 (385h4m20.019502217s) [upload=338812]: Done starting upload Oct 9 14:35:34.557620766 (385h4m20.019713249s) [upload=338812]: Uploading segments... Oct 9 14:35:34.557985310 (385h4m20.020077794s) [upload=338812]: Got next segment. Inline: false Oct 9 14:35:34.558047720 (385h4m20.020140203s) [upload=338812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:35:34.558054152 (385h4m20.020146635s) [upload=338812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:35:34.851664575 (385h4m20.313757058s) [seg_pos={0 0}, upload=338812]: Begin upload segment... Oct 9 14:35:34.851735049 (385h4m20.313827522s) [seg_pos={0 0}, upload=338812]: Done begin upload segment. Oct 9 14:35:34.851785275 (385h4m20.313877748s) [noise=true , address=145.79.227.15:4876 , piece_id=YVVJJBHMIGE5FIAK3GGRP42DGYS34OTAHGFYX7BI7O44YS2GCEAA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.851835971 (385h4m20.313928455s) [noise=true , address=45.91.116.10:4082 , piece_id=52SYFUAWBI6WDNXLYNVFGLYRLY2U3FDXQL3DIDQVM3WR2UD7EIXQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.852610968 (385h4m20.314703451s) [noise=true , address=45.91.116.6:4973 , piece_id=ASDBNYMP5L4GN3FMISWFEIV3NEXNJ2T6U6MWLMHSV3OEBGI3AKUA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.853058018 (385h4m20.315150511s) [noise=true , address=145.79.227.5:4221 , piece_id=D6WV7FSKH6RL4BSQOJBN76UEVB3SCR262Y346VJZSXR6NZGEUJSA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.853554344 (385h4m20.315646827s) [noise=true , address=145.79.227.17:4969 , piece_id=PWVY63YM6WRF4HCYRVK75JU5FRJADJHQHCLUWOALVS2AGXVNAIGQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.854203069 (385h4m20.316295562s) [noise=true , address=45.91.116.11:4119 , piece_id=6D4CRYAPYLFY2BV7RMSJ27K2KOLIIPEPF3D5UX75TVG5YCXZW4PQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.854672613 (385h4m20.316765107s) [noise=true , address=77.90.31.21:4878 , piece_id=XASU3BSBVIUPIAU5XKEKGICGN6YQEZXHOMJ3DVJVQNGRJF62PPOA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.855201921 (385h4m20.317294404s) [noise=true , address=45.91.116.13:4154 , piece_id=CQMUUX35QCYRXDHARK65OVTLXPFIC5VOFXXNQKLSKQG2WKF2F4KA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.856604713 (385h4m20.318697226s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.856722898 (385h4m20.318815371s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.858224687 (385h4m20.32031717s) [noise=true , address=145.79.227.11:4729 , piece_id=WR4662VAXJWHLVZJOWVRANINDEI3HTKA3HH4E7G77TUV2YQ7CIHQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.858568722 (385h4m20.320661205s) [noise=true , address=77.90.31.16:4734 , piece_id=5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.859568797 (385h4m20.32166128s) [noise=true , address=45.91.116.7:4010 , piece_id=WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.861282199 (385h4m20.323374682s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.861530292 (385h4m20.323622775s) [noise=true , address=45.91.116.4:4939 , piece_id=PPXXI6O35ZMSM5XAG32PEPMTOPKY4TBZDWCUVIWSM3GI672DTDUQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.862867228 (385h4m20.324959701s) [noise=true , address=77.90.31.17:4772 , piece_id=JBMSDHNS5OEOOQK2M6DVWAVBTMWJCNVIFHZGIMBZYVALCPFMLA5Q, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:36.795950136 (385h4m22.258042619s) [upload=338812]: Next returned nil segment Oct 9 14:35:36.795974834 (385h4m22.258067317s) [upload=338812]: Waiting for error group managing segments... Oct 9 14:35:37.127484366 (385h4m22.589576849s) [noise=true , address=45.91.116.4:4939 , piece_id=PPXXI6O35ZMSM5XAG32PEPMTOPKY4TBZDWCUVIWSM3GI672DTDUQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.128271093 (385h4m22.590363576s) [noise=true , address=45.91.116.11:4119 , piece_id=6D4CRYAPYLFY2BV7RMSJ27K2KOLIIPEPF3D5UX75TVG5YCXZW4PQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.233985636 (385h4m22.696078119s) [noise=true , address=45.91.116.13:4154 , piece_id=CQMUUX35QCYRXDHARK65OVTLXPFIC5VOFXXNQKLSKQG2WKF2F4KA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.277235002 (385h4m22.739327485s) [noise=true , address=45.91.116.7:4010 , piece_id=WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.308095466 (385h4m22.770187949s) [noise=true , address=45.91.116.6:4973 , piece_id=ASDBNYMP5L4GN3FMISWFEIV3NEXNJ2T6U6MWLMHSV3OEBGI3AKUA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.322292431 (385h4m22.784384914s) [noise=true , address=77.90.31.17:4772 , piece_id=JBMSDHNS5OEOOQK2M6DVWAVBTMWJCNVIFHZGIMBZYVALCPFMLA5Q, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.349608015 (385h4m22.811700498s) [noise=true , address=77.90.31.16:4734 , piece_id=5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.390237451 (385h4m22.852329934s) [noise=true , address=77.90.31.21:4878 , piece_id=XASU3BSBVIUPIAU5XKEKGICGN6YQEZXHOMJ3DVJVQNGRJF62PPOA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.612647666 (385h4m23.074740139s) [noise=true , address=45.91.116.10:4082 , piece_id=52SYFUAWBI6WDNXLYNVFGLYRLY2U3FDXQL3DIDQVM3WR2UD7EIXQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.655794517 (385h4m23.117887s) [noise=true , address=145.79.227.17:4969 , piece_id=PWVY63YM6WRF4HCYRVK75JU5FRJADJHQHCLUWOALVS2AGXVNAIGQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.797597779 (385h4m23.259690262s) [noise=true , address=145.79.227.5:4221 , piece_id=D6WV7FSKH6RL4BSQOJBN76UEVB3SCR262Y346VJZSXR6NZGEUJSA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.131909277 (385h4m23.59400177s) [noise=true , address=145.79.227.11:4729 , piece_id=WR4662VAXJWHLVZJOWVRANINDEI3HTKA3HH4E7G77TUV2YQ7CIHQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.228933852 (385h4m23.691026335s) [noise=true , address=145.79.227.15:4876 , piece_id=YVVJJBHMIGE5FIAK3GGRP42DGYS34OTAHGFYX7BI7O44YS2GCEAA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.228970822 (385h4m23.691063295s) [seg_pos={0 0}, upload=338812]: Segment reached optimal threshold of 13 pieces. Oct 9 14:35:38.229158590 (385h4m23.691251073s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229173057 (385h4m23.69126555s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229187265 (385h4m23.691279748s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229195140 (385h4m23.691287623s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229269261 (385h4m23.691361744s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229280762 (385h4m23.691373255s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229299208 (385h4m23.691391691s) [seg_pos={0 0}, upload=338812]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:35:38.229388447 (385h4m23.69148093s) [upload=338812]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:35:38.229395131 (385h4m23.691487614s) [upload=338812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:35:38.557592776 (385h4m24.019685249s) [upload=338812]: Done uploading segments... Oct 9 14:35:38.557614237 (385h4m24.01970672s) [upload=338812]: Upload finished. err: <nil> Duration: 24.021458244s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X.......X..............X...........X..X..X..X....X......X.X] duration: 30.003015874s FAIL (2025-10-09 14:36:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: 5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A) context deadline exceeded ======LOGS====== Oct 9 14:35:34.557314854 (385h4m20.019407337s) [upload=338812]: Starting upload Oct 9 14:35:34.557409734 (385h4m20.019502217s) [upload=338812]: Done starting upload Oct 9 14:35:34.557620766 (385h4m20.019713249s) [upload=338812]: Uploading segments... Oct 9 14:35:34.557985310 (385h4m20.020077794s) [upload=338812]: Got next segment. Inline: false Oct 9 14:35:34.558047720 (385h4m20.020140203s) [upload=338812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:35:34.558054152 (385h4m20.020146635s) [upload=338812]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:35:34.851664575 (385h4m20.313757058s) [seg_pos={0 0}, upload=338812]: Begin upload segment... Oct 9 14:35:34.851735049 (385h4m20.313827522s) [seg_pos={0 0}, upload=338812]: Done begin upload segment. Oct 9 14:35:34.851785275 (385h4m20.313877748s) [noise=true , address=145.79.227.15:4876 , piece_id=YVVJJBHMIGE5FIAK3GGRP42DGYS34OTAHGFYX7BI7O44YS2GCEAA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.851835971 (385h4m20.313928455s) [noise=true , address=45.91.116.10:4082 , piece_id=52SYFUAWBI6WDNXLYNVFGLYRLY2U3FDXQL3DIDQVM3WR2UD7EIXQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.852610968 (385h4m20.314703451s) [noise=true , address=45.91.116.6:4973 , piece_id=ASDBNYMP5L4GN3FMISWFEIV3NEXNJ2T6U6MWLMHSV3OEBGI3AKUA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.853058018 (385h4m20.315150511s) [noise=true , address=145.79.227.5:4221 , piece_id=D6WV7FSKH6RL4BSQOJBN76UEVB3SCR262Y346VJZSXR6NZGEUJSA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.853554344 (385h4m20.315646827s) [noise=true , address=145.79.227.17:4969 , piece_id=PWVY63YM6WRF4HCYRVK75JU5FRJADJHQHCLUWOALVS2AGXVNAIGQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.854203069 (385h4m20.316295562s) [noise=true , address=45.91.116.11:4119 , piece_id=6D4CRYAPYLFY2BV7RMSJ27K2KOLIIPEPF3D5UX75TVG5YCXZW4PQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.854672613 (385h4m20.316765107s) [noise=true , address=77.90.31.21:4878 , piece_id=XASU3BSBVIUPIAU5XKEKGICGN6YQEZXHOMJ3DVJVQNGRJF62PPOA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.855201921 (385h4m20.317294404s) [noise=true , address=45.91.116.13:4154 , piece_id=CQMUUX35QCYRXDHARK65OVTLXPFIC5VOFXXNQKLSKQG2WKF2F4KA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.856604713 (385h4m20.318697226s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.856722898 (385h4m20.318815371s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.858224687 (385h4m20.32031717s) [noise=true , address=145.79.227.11:4729 , piece_id=WR4662VAXJWHLVZJOWVRANINDEI3HTKA3HH4E7G77TUV2YQ7CIHQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.858568722 (385h4m20.320661205s) [noise=true , address=77.90.31.16:4734 , piece_id=5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.859568797 (385h4m20.32166128s) [noise=true , address=45.91.116.7:4010 , piece_id=WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.861282199 (385h4m20.323374682s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.861530292 (385h4m20.323622775s) [noise=true , address=45.91.116.4:4939 , piece_id=PPXXI6O35ZMSM5XAG32PEPMTOPKY4TBZDWCUVIWSM3GI672DTDUQ, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:34.862867228 (385h4m20.324959701s) [noise=true , address=77.90.31.17:4772 , piece_id=JBMSDHNS5OEOOQK2M6DVWAVBTMWJCNVIFHZGIMBZYVALCPFMLA5Q, seg_pos={0 0}, upload=338812]: Uploading piece... Oct 9 14:35:36.795950136 (385h4m22.258042619s) [upload=338812]: Next returned nil segment Oct 9 14:35:36.795974834 (385h4m22.258067317s) [upload=338812]: Waiting for error group managing segments... Oct 9 14:35:37.127484366 (385h4m22.589576849s) [noise=true , address=45.91.116.4:4939 , piece_id=PPXXI6O35ZMSM5XAG32PEPMTOPKY4TBZDWCUVIWSM3GI672DTDUQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.128271093 (385h4m22.590363576s) [noise=true , address=45.91.116.11:4119 , piece_id=6D4CRYAPYLFY2BV7RMSJ27K2KOLIIPEPF3D5UX75TVG5YCXZW4PQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.233985636 (385h4m22.696078119s) [noise=true , address=45.91.116.13:4154 , piece_id=CQMUUX35QCYRXDHARK65OVTLXPFIC5VOFXXNQKLSKQG2WKF2F4KA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.277235002 (385h4m22.739327485s) [noise=true , address=45.91.116.7:4010 , piece_id=WEI5HIVSUIHU2KDJ2RBFZHTO3STL5WY5W5Q2QFKT7LQDHD5TM54A, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.308095466 (385h4m22.770187949s) [noise=true , address=45.91.116.6:4973 , piece_id=ASDBNYMP5L4GN3FMISWFEIV3NEXNJ2T6U6MWLMHSV3OEBGI3AKUA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.322292431 (385h4m22.784384914s) [noise=true , address=77.90.31.17:4772 , piece_id=JBMSDHNS5OEOOQK2M6DVWAVBTMWJCNVIFHZGIMBZYVALCPFMLA5Q, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.349608015 (385h4m22.811700498s) [noise=true , address=77.90.31.16:4734 , piece_id=5KHCX4MNROAP2QVUTIIKLQZK4CZXW6J2QM5AMURZOJKX7UF2EFUA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.390237451 (385h4m22.852329934s) [noise=true , address=77.90.31.21:4878 , piece_id=XASU3BSBVIUPIAU5XKEKGICGN6YQEZXHOMJ3DVJVQNGRJF62PPOA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.612647666 (385h4m23.074740139s) [noise=true , address=45.91.116.10:4082 , piece_id=52SYFUAWBI6WDNXLYNVFGLYRLY2U3FDXQL3DIDQVM3WR2UD7EIXQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.655794517 (385h4m23.117887s) [noise=true , address=145.79.227.17:4969 , piece_id=PWVY63YM6WRF4HCYRVK75JU5FRJADJHQHCLUWOALVS2AGXVNAIGQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:37.797597779 (385h4m23.259690262s) [noise=true , address=145.79.227.5:4221 , piece_id=D6WV7FSKH6RL4BSQOJBN76UEVB3SCR262Y346VJZSXR6NZGEUJSA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.131909277 (385h4m23.59400177s) [noise=true , address=145.79.227.11:4729 , piece_id=WR4662VAXJWHLVZJOWVRANINDEI3HTKA3HH4E7G77TUV2YQ7CIHQ, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.228933852 (385h4m23.691026335s) [noise=true , address=145.79.227.15:4876 , piece_id=YVVJJBHMIGE5FIAK3GGRP42DGYS34OTAHGFYX7BI7O44YS2GCEAA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: <nil> Oct 9 14:35:38.228970822 (385h4m23.691063295s) [seg_pos={0 0}, upload=338812]: Segment reached optimal threshold of 13 pieces. Oct 9 14:35:38.229158590 (385h4m23.691251073s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229173057 (385h4m23.69126555s) [noise=true , address=145.79.227.16:4910 , piece_id=MXZEAOHVFSAI6T5Q3K5VWLDHKC45F74USDDDZHJKWBBCODDUUKYA, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229187265 (385h4m23.691279748s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229195140 (385h4m23.691287623s) [noise=true , address=145.79.227.2:4115 , piece_id=U2KHHMOUNQIHHL43XTOTEPTDACOHU5FB4U7UCN4HWBNMCOBQ2U6Q, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229269261 (385h4m23.691361744s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:35:38.229280762 (385h4m23.691373255s) [noise=true , address=145.79.227.13:4804 , piece_id=Z56ZHXUV2CIADY6BHLWDFCKPOYWOFLSNFWYK43KZKJIMUMZPYH7A, seg_pos={0 0}, upload=338812]: Long tail cancellation due to optimal threshold Oct 9 14:35:38.229299208 (385h4m23.691391691s) [seg_pos={0 0}, upload=338812]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:35:38.229388447 (385h4m23.69148093s) [upload=338812]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:35:38.229395131 (385h4m23.691487614s) [upload=338812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:35:38.557592776 (385h4m24.019685249s) [upload=338812]: Done uploading segments... Oct 9 14:35:38.557614237 (385h4m24.01970672s) [upload=338812]: Upload finished. err: <nil> Duration: 30.003043517s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..X.......X..............X...........X..X..X..X....X......X.] duration: 21.984294726s FAIL (2025-10-09 14:35:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:34:05.986878414 (385h2m51.448970897s) [upload=338791]: Starting upload Oct 9 14:34:05.986977793 (385h2m51.449070266s) [upload=338791]: Done starting upload Oct 9 14:34:05.987140833 (385h2m51.449233316s) [upload=338791]: Uploading segments... Oct 9 14:34:05.987291410 (385h2m51.449383893s) [upload=338791]: Got next segment. Inline: false Oct 9 14:34:05.987345923 (385h2m51.449438406s) [upload=338791]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:34:05.987352035 (385h2m51.449444518s) [upload=338791]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:34:06.648063535 (385h2m52.110156028s) [seg_pos={0 0}, upload=338791]: Begin upload segment... Oct 9 14:34:06.648145762 (385h2m52.110238245s) [seg_pos={0 0}, upload=338791]: Done begin upload segment. Oct 9 14:34:06.648173595 (385h2m52.110266078s) [noise=true , address=77.90.31.14:4661 , piece_id=EXDAEQNBNM75C4W3ZJCRC2M6VF4BWCXF5MSZQN7SEAP26UCCFAKQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.648859861 (385h2m52.110952344s) [noise=true , address=145.79.227.11:4729 , piece_id=G5RPMGNDZANSX7CBLFN4HYO4KPLSVXFGK2CBBGJ4HHKSAXQGCSOQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.649285552 (385h2m52.111378035s) [noise=true , address=45.91.116.6:4975 , piece_id=7PUM24MCUD2WEOVL7TY2UZCMXKO4J5VDZR2XU3PKG4LOD4UQ4YWQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650002467 (385h2m52.11209494s) [noise=true , address=45.91.116.14:4191 , piece_id=DPDCOV2IECYZD6R3TJWRK4YP7BGVVJFPTAV2IRWUCWH4VFYHTONA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650328338 (385h2m52.112420821s) [noise=true , address=145.79.227.16:4909 , piece_id=UTNHPW55N5VSTRMWICODUWLYYTU5SKQTVWZH56BLTZG7TOXHYVKA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650631735 (385h2m52.112724208s) [noise=true , address=145.79.227.14:4838 , piece_id=7AZYD63RIUXNN5RUVNABFY5WLQRGIJ23DQ6GMGCNTZL2WLBBUE4Q, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650956815 (385h2m52.113049298s) [noise=true , address=145.79.227.18:4006 , piece_id=73AU4B6JU5QUPWTBHMJXK7OHYMHDGKI7D23Q4KHLGGYXDHPTO25Q, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.651380972 (385h2m52.113473455s) [noise=true , address=45.91.116.9:4263 , piece_id=UG3NFVBIGPO7G4FWGN7L2UBOHSTKSUL7RCF3IN3G4DC5YP2DTT7A, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.651796934 (385h2m52.113889407s) [noise=true , address=145.79.227.5:4222 , piece_id=5XOWAFIBQ4HKEDXTDO4SEV5NK25CIURPTWUFO7PNTG5WOBFPN6YQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.652227574 (385h2m52.114320047s) [noise=true , address=45.91.116.12:4226 , piece_id=BAO7NYW4UGCWOYOWKDWZWREJJMV4SFYFQF36L57WRIBGBNBYQUGQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.653705318 (385h2m52.115797801s) [noise=true , address=77.90.31.19:4841 , piece_id=CIFIBCOP4UFZBOJ2OLEXHX4RVX26Y4SAK4WZPED66YQ2KYTFZTRA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.653834164 (385h2m52.115926677s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.655334390 (385h2m52.117426874s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.655407991 (385h2m52.117500474s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.656845137 (385h2m52.11893761s) [noise=true , address=77.90.31.11:4554 , piece_id=DSC4X5FEEB3TP5O2GR7B4EBIGMIYPDCRDWM6VJYQC37AECOOMYLA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.656928887 (385h2m52.11902137s) [noise=true , address=145.79.227.11:4731 , piece_id=DZYPIDR22AUGBAZVTKOR6IMHHPPDV3YJHU7N223LUJY5G2X42KJA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:09.050277586 (385h2m54.512370069s) [upload=338791]: Next returned nil segment Oct 9 14:34:09.050299717 (385h2m54.5123922s) [upload=338791]: Waiting for error group managing segments... Oct 9 14:34:09.412863594 (385h2m54.874956077s) [noise=true , address=45.91.116.9:4263 , piece_id=UG3NFVBIGPO7G4FWGN7L2UBOHSTKSUL7RCF3IN3G4DC5YP2DTT7A, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.523671374 (385h2m54.985763857s) [noise=true , address=45.91.116.6:4975 , piece_id=7PUM24MCUD2WEOVL7TY2UZCMXKO4J5VDZR2XU3PKG4LOD4UQ4YWQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.575065289 (385h2m55.037157772s) [noise=true , address=145.79.227.5:4222 , piece_id=5XOWAFIBQ4HKEDXTDO4SEV5NK25CIURPTWUFO7PNTG5WOBFPN6YQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.589233659 (385h2m55.051326142s) [noise=true , address=145.79.227.11:4731 , piece_id=DZYPIDR22AUGBAZVTKOR6IMHHPPDV3YJHU7N223LUJY5G2X42KJA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.649757388 (385h2m55.111849871s) [noise=true , address=145.79.227.16:4909 , piece_id=UTNHPW55N5VSTRMWICODUWLYYTU5SKQTVWZH56BLTZG7TOXHYVKA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.711217441 (385h2m55.173309914s) [noise=true , address=145.79.227.18:4006 , piece_id=73AU4B6JU5QUPWTBHMJXK7OHYMHDGKI7D23Q4KHLGGYXDHPTO25Q, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.761229665 (385h2m55.223322148s) [noise=true , address=45.91.116.14:4191 , piece_id=DPDCOV2IECYZD6R3TJWRK4YP7BGVVJFPTAV2IRWUCWH4VFYHTONA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.777547267 (385h2m55.23963975s) [noise=true , address=77.90.31.19:4841 , piece_id=CIFIBCOP4UFZBOJ2OLEXHX4RVX26Y4SAK4WZPED66YQ2KYTFZTRA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.829776041 (385h2m55.291868524s) [noise=true , address=145.79.227.11:4729 , piece_id=G5RPMGNDZANSX7CBLFN4HYO4KPLSVXFGK2CBBGJ4HHKSAXQGCSOQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.874594655 (385h2m55.336687138s) [noise=true , address=77.90.31.11:4554 , piece_id=DSC4X5FEEB3TP5O2GR7B4EBIGMIYPDCRDWM6VJYQC37AECOOMYLA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.877090738 (385h2m55.339183231s) [noise=true , address=77.90.31.14:4661 , piece_id=EXDAEQNBNM75C4W3ZJCRC2M6VF4BWCXF5MSZQN7SEAP26UCCFAKQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.898066912 (385h2m55.360159395s) [noise=true , address=145.79.227.14:4838 , piece_id=7AZYD63RIUXNN5RUVNABFY5WLQRGIJ23DQ6GMGCNTZL2WLBBUE4Q, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.905205164 (385h2m55.367297637s) [noise=true , address=45.91.116.12:4226 , piece_id=BAO7NYW4UGCWOYOWKDWZWREJJMV4SFYFQF36L57WRIBGBNBYQUGQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.905232426 (385h2m55.367324899s) [seg_pos={0 0}, upload=338791]: Segment reached optimal threshold of 13 pieces. Oct 9 14:34:09.905371371 (385h2m55.367463854s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905381500 (385h2m55.367473983s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905482703 (385h2m55.367575186s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905488584 (385h2m55.367581067s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905570098 (385h2m55.367662571s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905578394 (385h2m55.367670877s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905595367 (385h2m55.36768785s) [seg_pos={0 0}, upload=338791]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:34:09.905691350 (385h2m55.367783833s) [upload=338791]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:34:09.905702060 (385h2m55.367794543s) [upload=338791]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:34:10.269026334 (385h2m55.731118817s) [upload=338791]: Done uploading segments... Oct 9 14:34:10.269054748 (385h2m55.731147231s) [upload=338791]: Upload finished. err: <nil> Duration: 21.984325554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...X.......X..............X...........X..X..X..X....X......X] duration: 30.002058009s FAIL (2025-10-09 14:34:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:34:05.986878414 (385h2m51.448970897s) [upload=338791]: Starting upload Oct 9 14:34:05.986977793 (385h2m51.449070266s) [upload=338791]: Done starting upload Oct 9 14:34:05.987140833 (385h2m51.449233316s) [upload=338791]: Uploading segments... Oct 9 14:34:05.987291410 (385h2m51.449383893s) [upload=338791]: Got next segment. Inline: false Oct 9 14:34:05.987345923 (385h2m51.449438406s) [upload=338791]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:34:05.987352035 (385h2m51.449444518s) [upload=338791]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:34:06.648063535 (385h2m52.110156028s) [seg_pos={0 0}, upload=338791]: Begin upload segment... Oct 9 14:34:06.648145762 (385h2m52.110238245s) [seg_pos={0 0}, upload=338791]: Done begin upload segment. Oct 9 14:34:06.648173595 (385h2m52.110266078s) [noise=true , address=77.90.31.14:4661 , piece_id=EXDAEQNBNM75C4W3ZJCRC2M6VF4BWCXF5MSZQN7SEAP26UCCFAKQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.648859861 (385h2m52.110952344s) [noise=true , address=145.79.227.11:4729 , piece_id=G5RPMGNDZANSX7CBLFN4HYO4KPLSVXFGK2CBBGJ4HHKSAXQGCSOQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.649285552 (385h2m52.111378035s) [noise=true , address=45.91.116.6:4975 , piece_id=7PUM24MCUD2WEOVL7TY2UZCMXKO4J5VDZR2XU3PKG4LOD4UQ4YWQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650002467 (385h2m52.11209494s) [noise=true , address=45.91.116.14:4191 , piece_id=DPDCOV2IECYZD6R3TJWRK4YP7BGVVJFPTAV2IRWUCWH4VFYHTONA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650328338 (385h2m52.112420821s) [noise=true , address=145.79.227.16:4909 , piece_id=UTNHPW55N5VSTRMWICODUWLYYTU5SKQTVWZH56BLTZG7TOXHYVKA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650631735 (385h2m52.112724208s) [noise=true , address=145.79.227.14:4838 , piece_id=7AZYD63RIUXNN5RUVNABFY5WLQRGIJ23DQ6GMGCNTZL2WLBBUE4Q, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.650956815 (385h2m52.113049298s) [noise=true , address=145.79.227.18:4006 , piece_id=73AU4B6JU5QUPWTBHMJXK7OHYMHDGKI7D23Q4KHLGGYXDHPTO25Q, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.651380972 (385h2m52.113473455s) [noise=true , address=45.91.116.9:4263 , piece_id=UG3NFVBIGPO7G4FWGN7L2UBOHSTKSUL7RCF3IN3G4DC5YP2DTT7A, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.651796934 (385h2m52.113889407s) [noise=true , address=145.79.227.5:4222 , piece_id=5XOWAFIBQ4HKEDXTDO4SEV5NK25CIURPTWUFO7PNTG5WOBFPN6YQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.652227574 (385h2m52.114320047s) [noise=true , address=45.91.116.12:4226 , piece_id=BAO7NYW4UGCWOYOWKDWZWREJJMV4SFYFQF36L57WRIBGBNBYQUGQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.653705318 (385h2m52.115797801s) [noise=true , address=77.90.31.19:4841 , piece_id=CIFIBCOP4UFZBOJ2OLEXHX4RVX26Y4SAK4WZPED66YQ2KYTFZTRA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.653834164 (385h2m52.115926677s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.655334390 (385h2m52.117426874s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.655407991 (385h2m52.117500474s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.656845137 (385h2m52.11893761s) [noise=true , address=77.90.31.11:4554 , piece_id=DSC4X5FEEB3TP5O2GR7B4EBIGMIYPDCRDWM6VJYQC37AECOOMYLA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:06.656928887 (385h2m52.11902137s) [noise=true , address=145.79.227.11:4731 , piece_id=DZYPIDR22AUGBAZVTKOR6IMHHPPDV3YJHU7N223LUJY5G2X42KJA, seg_pos={0 0}, upload=338791]: Uploading piece... Oct 9 14:34:09.050277586 (385h2m54.512370069s) [upload=338791]: Next returned nil segment Oct 9 14:34:09.050299717 (385h2m54.5123922s) [upload=338791]: Waiting for error group managing segments... Oct 9 14:34:09.412863594 (385h2m54.874956077s) [noise=true , address=45.91.116.9:4263 , piece_id=UG3NFVBIGPO7G4FWGN7L2UBOHSTKSUL7RCF3IN3G4DC5YP2DTT7A, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.523671374 (385h2m54.985763857s) [noise=true , address=45.91.116.6:4975 , piece_id=7PUM24MCUD2WEOVL7TY2UZCMXKO4J5VDZR2XU3PKG4LOD4UQ4YWQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.575065289 (385h2m55.037157772s) [noise=true , address=145.79.227.5:4222 , piece_id=5XOWAFIBQ4HKEDXTDO4SEV5NK25CIURPTWUFO7PNTG5WOBFPN6YQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.589233659 (385h2m55.051326142s) [noise=true , address=145.79.227.11:4731 , piece_id=DZYPIDR22AUGBAZVTKOR6IMHHPPDV3YJHU7N223LUJY5G2X42KJA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.649757388 (385h2m55.111849871s) [noise=true , address=145.79.227.16:4909 , piece_id=UTNHPW55N5VSTRMWICODUWLYYTU5SKQTVWZH56BLTZG7TOXHYVKA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.711217441 (385h2m55.173309914s) [noise=true , address=145.79.227.18:4006 , piece_id=73AU4B6JU5QUPWTBHMJXK7OHYMHDGKI7D23Q4KHLGGYXDHPTO25Q, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.761229665 (385h2m55.223322148s) [noise=true , address=45.91.116.14:4191 , piece_id=DPDCOV2IECYZD6R3TJWRK4YP7BGVVJFPTAV2IRWUCWH4VFYHTONA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.777547267 (385h2m55.23963975s) [noise=true , address=77.90.31.19:4841 , piece_id=CIFIBCOP4UFZBOJ2OLEXHX4RVX26Y4SAK4WZPED66YQ2KYTFZTRA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.829776041 (385h2m55.291868524s) [noise=true , address=145.79.227.11:4729 , piece_id=G5RPMGNDZANSX7CBLFN4HYO4KPLSVXFGK2CBBGJ4HHKSAXQGCSOQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.874594655 (385h2m55.336687138s) [noise=true , address=77.90.31.11:4554 , piece_id=DSC4X5FEEB3TP5O2GR7B4EBIGMIYPDCRDWM6VJYQC37AECOOMYLA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.877090738 (385h2m55.339183231s) [noise=true , address=77.90.31.14:4661 , piece_id=EXDAEQNBNM75C4W3ZJCRC2M6VF4BWCXF5MSZQN7SEAP26UCCFAKQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.898066912 (385h2m55.360159395s) [noise=true , address=145.79.227.14:4838 , piece_id=7AZYD63RIUXNN5RUVNABFY5WLQRGIJ23DQ6GMGCNTZL2WLBBUE4Q, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.905205164 (385h2m55.367297637s) [noise=true , address=45.91.116.12:4226 , piece_id=BAO7NYW4UGCWOYOWKDWZWREJJMV4SFYFQF36L57WRIBGBNBYQUGQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: <nil> Oct 9 14:34:09.905232426 (385h2m55.367324899s) [seg_pos={0 0}, upload=338791]: Segment reached optimal threshold of 13 pieces. Oct 9 14:34:09.905371371 (385h2m55.367463854s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905381500 (385h2m55.367473983s) [noise=true , address=77.90.31.21:4878 , piece_id=LQJK4VOFBK7B2RYN2HKWDF7AOHPUAEJJEWMXLL2FTJDBC2RAGEEQ, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905482703 (385h2m55.367575186s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905488584 (385h2m55.367581067s) [noise=true , address=145.79.227.15:4874 , piece_id=GMMXORSV65WUQIB3PD76HA2O5MJ6D3ATUNT2LI37HCIS3NAZ3DVA, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905570098 (385h2m55.367662571s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 9 14:34:09.905578394 (385h2m55.367670877s) [noise=true , address=77.90.31.14:4663 , piece_id=XRC2ZLXUPWQO2CZDR2XSV3KOU5ROSHT447P5QJ5PVNCPVBRVINAA, seg_pos={0 0}, upload=338791]: Long tail cancellation due to optimal threshold Oct 9 14:34:09.905595367 (385h2m55.36768785s) [seg_pos={0 0}, upload=338791]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:34:09.905691350 (385h2m55.367783833s) [upload=338791]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:34:09.905702060 (385h2m55.367794543s) [upload=338791]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:34:10.269026334 (385h2m55.731118817s) [upload=338791]: Done uploading segments... Oct 9 14:34:10.269054748 (385h2m55.731147231s) [upload=338791]: Upload finished. err: <nil> Duration: 30.00208432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....X.......X..............X...........X..X..X..X....X......] duration: 28.865213327s FAIL (2025-10-09 14:33:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 28.865235079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....X.......X..............X...........X..X..X..X....X.....] duration: 25.543734282s FAIL (2025-10-09 14:32:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 25.54377471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [......X.......X..............X...........X..X..X..X....X....] duration: 20.992644127s FAIL (2025-10-09 14:31:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 20.992680999s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......X.......X..............X...........X..X..X..X....X...] duration: 17.167900669s FAIL (2025-10-09 14:31:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 17.167935083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [........X.......X..............X...........X..X..X..X....X..] duration: 27.240533942s FAIL (2025-10-09 14:30:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 27.240570813s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........X.......X..............X...........X..X..X..X....X.] duration: 24.746711423s FAIL (2025-10-09 14:30:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 24.746737653s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........X.......X..............X...........X..X..X..X....X] duration: 30.001666726s FAIL (2025-10-09 14:29:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:29:11.424852870 (384h57m56.886945353s) [upload=338718]: Starting upload Oct 9 14:29:11.424957339 (384h57m56.887049822s) [upload=338718]: Done starting upload Oct 9 14:29:11.425041019 (384h57m56.887133502s) [upload=338718]: Uploading segments... Oct 9 14:29:11.425325050 (384h57m56.887417523s) [upload=338718]: Got next segment. Inline: false Oct 9 14:29:11.425366789 (384h57m56.887459272s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:29:11.425372420 (384h57m56.887464903s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:29:11.680173592 (384h57m57.142266075s) [seg_pos={0 0}, upload=338718]: Begin upload segment... Oct 9 14:29:11.680225912 (384h57m57.142318385s) [seg_pos={0 0}, upload=338718]: Done begin upload segment. Oct 9 14:29:11.680251430 (384h57m57.142343913s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680277761 (384h57m57.142370244s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.680939461 (384h57m57.143031944s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681017599 (384h57m57.143110082s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681399987 (384h57m57.143492461s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.681756155 (384h57m57.143848639s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682187147 (384h57m57.14427963s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682198167 (384h57m57.14429065s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682484542 (384h57m57.144577026s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682575195 (384h57m57.144667679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682731544 (384h57m57.144824027s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.682966080 (384h57m57.145058553s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683051232 (384h57m57.145143705s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683328941 (384h57m57.145421414s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.683385148 (384h57m57.145477631s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:11.685172872 (384h57m57.147265355s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Uploading piece... Oct 9 14:29:13.551981837 (384h57m59.01407431s) [upload=338718]: Next returned nil segment Oct 9 14:29:13.552003419 (384h57m59.014095922s) [upload=338718]: Waiting for error group managing segments... Oct 9 14:29:14.084845651 (384h57m59.546938134s) [noise=true , address=45.91.116.6:4973 , piece_id=GUQOZGBF2HDU67FSMWHOW5K6WRATFB64TNRERTJMVCFU2YDWOZFQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.149560101 (384h57m59.611652584s) [noise=true , address=45.91.116.13:4153 , piece_id=ZPGZXCKL3JV5OT6YIKWJEK3PRJEOOVOUV5SJIMTBSL6IFXTLDABQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.155490393 (384h57m59.617582886s) [noise=true , address=45.91.116.5:4297 , piece_id=E3SM6YDXOAKGLHCLBMHL5B5UN3TJPZ4R4KDVCIMUNVJ3F7DJ4HYQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.263007277 (384h57m59.72509976s) [noise=true , address=45.91.116.7:4010 , piece_id=ETORV5ELUPO7YRCS7J2YTKD7EKYANAMHNFFC3G7IF4LGR24Y5P7A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.358014942 (384h57m59.820107425s) [noise=true , address=45.91.116.11:4118 , piece_id=WO2YR3DDBXDMNFC2LLNRFH5QVR4S4PGXKC2SLZL4UKQJKIF5CIAA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.599800962 (384h58m0.061893445s) [noise=true , address=145.79.227.5:4222 , piece_id=NVYEX4N7OQLQPFLVXRC5YLUFZ26OMCLJKJLICQBUYLS5KQKJD5AA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.719085173 (384h58m0.181177656s) [noise=true , address=145.79.227.4:4185 , piece_id=2QRVNWKXZRAR2NF4MV565HVNVYTR6A7G6DELIVUESYKCEM3RAWZQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.730094408 (384h58m0.192186881s) [noise=true , address=145.79.227.17:4972 , piece_id=CVV5JDQSEK4YT5I4TVZLMZGRMNDD4Y2TIHWZSDFEQZG4WJMOHV3A, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:14.974718752 (384h58m0.436811245s) [noise=true , address=45.91.116.4:4938 , piece_id=NFCKXMJE2MKC7CQZCEHUENDG2PHINWBYZRAIP64JBXDZPCIBC7DQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.094934699 (384h58m0.557027182s) [noise=true , address=77.90.31.12:4592 , piece_id=2UBY2ZH23TMMUVRDO6CQJCQUJ77CKIVQTFM5K4GPJWYLCDVBEKVQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.438467616 (384h58m0.900560089s) [noise=true , address=145.79.227.14:4838 , piece_id=ZYGF6QTTQPI2PULDNTNESORKYG7A3KZOD3UF3TWSUOBELYKFLZ2Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.765574436 (384h58m1.227666919s) [noise=true , address=145.79.227.5:4223 , piece_id=PUDR2DXOGSIQYAPAUUUBN5KY7RX7TQ7MEHQZITIPW3KQ4G2NFG4Q, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785125034 (384h58m1.247217517s) [noise=true , address=45.91.116.13:4154 , piece_id=P72REYJAFWHT2F62H3UYLK4ULRCOUMRAWSYWHYJLY6RB7GK2KJIQ, seg_pos={0 0}, upload=338718]: Done uploading piece. err: <nil> Oct 9 14:29:15.785162776 (384h58m1.247255249s) [seg_pos={0 0}, upload=338718]: Segment reached optimal threshold of 13 pieces. Oct 9 14:29:15.785298815 (384h58m1.247391298s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785309376 (384h58m1.247401859s) [noise=true , address=145.79.227.16:4910 , piece_id=MRZ5WL4QHQW6DG7FLGS5EQSOKTNM4HJ7OM6HUJODJTUYI3KJ6FWA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785385791 (384h58m1.247478274s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785392674 (384h58m1.247485157s) [noise=true , address=145.79.227.12:4767 , piece_id=RDRR7YZDCDATJHKB6X77JPAPPDMXSHTZT5JIOVDWCVHGHVNFBULA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785466425 (384h58m1.247558918s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF Oct 9 14:29:15.785491372 (384h58m1.247583845s) [noise=true , address=145.79.227.13:4804 , piece_id=PQNN7ZUORGWFQQH5OAPWZMUNBBRJMNNZFL6PW7VUV6ISMMZ4DFSA, seg_pos={0 0}, upload=338718]: Long tail cancellation due to optimal threshold Oct 9 14:29:15.785549913 (384h58m1.247642396s) [seg_pos={0 0}, upload=338718]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:29:15.785716310 (384h58m1.247808783s) [upload=338718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:29:15.785725448 (384h58m1.247817932s) [upload=338718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:29:16.101218963 (384h58m1.563311457s) [upload=338718]: Done uploading segments... Oct 9 14:29:16.101302031 (384h58m1.563394514s) [upload=338718]: Upload finished. err: <nil> Duration: 30.001704098s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...........X.......X..............X...........X..X..X..X....] duration: 13.3138284s FAIL (2025-10-09 14:29:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA) context deadline exceeded; piecestore: (Node ID: 1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, Piece ID: OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ) context deadline exceeded ======LOGS====== ct 9 14:26:02.931066356 (384h54m48.393158849s) [upload=338672]: Starting upload Oct 9 14:26:02.931150146 (384h54m48.393242629s) [upload=338672]: Done starting upload Oct 9 14:26:02.931263381 (384h54m48.393355864s) [upload=338672]: Uploading segments... Oct 9 14:26:02.931524760 (384h54m48.393617243s) [upload=338672]: Got next segment. Inline: false Oct 9 14:26:02.931579423 (384h54m48.393671906s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:26:02.931586807 (384h54m48.39367928s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:26:03.221262760 (384h54m48.683355253s) [seg_pos={0 0}, upload=338672]: Begin upload segment... Oct 9 14:26:03.221340026 (384h54m48.68343251s) [seg_pos={0 0}, upload=338672]: Done begin upload segment. Oct 9 14:26:03.221378089 (384h54m48.683470572s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.221423807 (384h54m48.68351629s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222245280 (384h54m48.684337773s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222946065 (384h54m48.685038549s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.223607265 (384h54m48.685699748s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.224204682 (384h54m48.686297165s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.225038119 (384h54m48.687130602s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.227192381 (384h54m48.689284864s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.230488067 (384h54m48.69258058s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.231261710 (384h54m48.693354193s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.235934918 (384h54m48.698027401s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.237597444 (384h54m48.699689937s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.240783411 (384h54m48.702875905s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.243963897 (384h54m48.70605638s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.247313737 (384h54m48.70940622s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.252695223 (384h54m48.714787706s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:05.442841804 (384h54m50.904934287s) [upload=338672]: Next returned nil segment Oct 9 14:26:05.442864387 (384h54m50.90495687s) [upload=338672]: Waiting for error group managing segments... Oct 9 14:26:05.934192787 (384h54m51.39628527s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.967835299 (384h54m51.429927783s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.974753102 (384h54m51.436845586s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.010380625 (384h54m51.472473118s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.021909868 (384h54m51.484002351s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.027655419 (384h54m51.489747902s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.038944626 (384h54m51.501037109s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.095896374 (384h54m51.557988847s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.102013522 (384h54m51.564106006s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.128550885 (384h54m51.590643368s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.134822928 (384h54m51.596915411s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.255075143 (384h54m51.717167626s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345427703 (384h54m51.807520176s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345465595 (384h54m51.807558078s) [seg_pos={0 0}, upload=338672]: Segment reached optimal threshold of 13 pieces. Oct 9 14:26:06.345686406 (384h54m51.807778899s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection Oct 9 14:26:06.345701043 (384h54m51.807793526s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345718196 (384h54m51.807810679s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345741059 (384h54m51.807833542s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345868813 (384h54m51.807961296s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345879122 (384h54m51.807971605s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345921412 (384h54m51.808013886s) [seg_pos={0 0}, upload=338672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:26:06.346037093 (384h54m51.808129577s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:26:06.346044117 (384h54m51.8081366s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:26:06.696762736 (384h54m52.158855219s) [upload=338672]: Done uploading segments... Oct 9 14:26:06.696787322 (384h54m52.158879796s) [upload=338672]: Upload finished. err: <nil> Duration: 13.313863737s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [............X.......X..............X...........X..X..X..X...] duration: 17.106276478s FAIL (2025-10-09 14:28:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA) context deadline exceeded; piecestore: (Node ID: 1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, Piece ID: OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ) context deadline exceeded ======LOGS====== ct 9 14:26:02.931066356 (384h54m48.393158849s) [upload=338672]: Starting upload Oct 9 14:26:02.931150146 (384h54m48.393242629s) [upload=338672]: Done starting upload Oct 9 14:26:02.931263381 (384h54m48.393355864s) [upload=338672]: Uploading segments... Oct 9 14:26:02.931524760 (384h54m48.393617243s) [upload=338672]: Got next segment. Inline: false Oct 9 14:26:02.931579423 (384h54m48.393671906s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:26:02.931586807 (384h54m48.39367928s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:26:03.221262760 (384h54m48.683355253s) [seg_pos={0 0}, upload=338672]: Begin upload segment... Oct 9 14:26:03.221340026 (384h54m48.68343251s) [seg_pos={0 0}, upload=338672]: Done begin upload segment. Oct 9 14:26:03.221378089 (384h54m48.683470572s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.221423807 (384h54m48.68351629s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222245280 (384h54m48.684337773s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222946065 (384h54m48.685038549s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.223607265 (384h54m48.685699748s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.224204682 (384h54m48.686297165s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.225038119 (384h54m48.687130602s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.227192381 (384h54m48.689284864s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.230488067 (384h54m48.69258058s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.231261710 (384h54m48.693354193s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.235934918 (384h54m48.698027401s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.237597444 (384h54m48.699689937s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.240783411 (384h54m48.702875905s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.243963897 (384h54m48.70605638s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.247313737 (384h54m48.70940622s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.252695223 (384h54m48.714787706s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:05.442841804 (384h54m50.904934287s) [upload=338672]: Next returned nil segment Oct 9 14:26:05.442864387 (384h54m50.90495687s) [upload=338672]: Waiting for error group managing segments... Oct 9 14:26:05.934192787 (384h54m51.39628527s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.967835299 (384h54m51.429927783s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.974753102 (384h54m51.436845586s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.010380625 (384h54m51.472473118s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.021909868 (384h54m51.484002351s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.027655419 (384h54m51.489747902s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.038944626 (384h54m51.501037109s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.095896374 (384h54m51.557988847s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.102013522 (384h54m51.564106006s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.128550885 (384h54m51.590643368s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.134822928 (384h54m51.596915411s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.255075143 (384h54m51.717167626s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345427703 (384h54m51.807520176s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345465595 (384h54m51.807558078s) [seg_pos={0 0}, upload=338672]: Segment reached optimal threshold of 13 pieces. Oct 9 14:26:06.345686406 (384h54m51.807778899s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection Oct 9 14:26:06.345701043 (384h54m51.807793526s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345718196 (384h54m51.807810679s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345741059 (384h54m51.807833542s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345868813 (384h54m51.807961296s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345879122 (384h54m51.807971605s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345921412 (384h54m51.808013886s) [seg_pos={0 0}, upload=338672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:26:06.346037093 (384h54m51.808129577s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:26:06.346044117 (384h54m51.8081366s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:26:06.696762736 (384h54m52.158855219s) [upload=338672]: Done uploading segments... Oct 9 14:26:06.696787322 (384h54m52.158879796s) [upload=338672]: Upload finished. err: <nil> Duration: 17.106340388s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X............X.......X..............X...........X..X..X..X..] duration: 11.459848486s FAIL (2025-10-09 14:27:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA) context deadline exceeded; piecestore: (Node ID: 1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, Piece ID: OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ) context deadline exceeded ======LOGS====== ct 9 14:26:02.931066356 (384h54m48.393158849s) [upload=338672]: Starting upload Oct 9 14:26:02.931150146 (384h54m48.393242629s) [upload=338672]: Done starting upload Oct 9 14:26:02.931263381 (384h54m48.393355864s) [upload=338672]: Uploading segments... Oct 9 14:26:02.931524760 (384h54m48.393617243s) [upload=338672]: Got next segment. Inline: false Oct 9 14:26:02.931579423 (384h54m48.393671906s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:26:02.931586807 (384h54m48.39367928s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:26:03.221262760 (384h54m48.683355253s) [seg_pos={0 0}, upload=338672]: Begin upload segment... Oct 9 14:26:03.221340026 (384h54m48.68343251s) [seg_pos={0 0}, upload=338672]: Done begin upload segment. Oct 9 14:26:03.221378089 (384h54m48.683470572s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.221423807 (384h54m48.68351629s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222245280 (384h54m48.684337773s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222946065 (384h54m48.685038549s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.223607265 (384h54m48.685699748s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.224204682 (384h54m48.686297165s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.225038119 (384h54m48.687130602s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.227192381 (384h54m48.689284864s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.230488067 (384h54m48.69258058s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.231261710 (384h54m48.693354193s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.235934918 (384h54m48.698027401s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.237597444 (384h54m48.699689937s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.240783411 (384h54m48.702875905s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.243963897 (384h54m48.70605638s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.247313737 (384h54m48.70940622s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.252695223 (384h54m48.714787706s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:05.442841804 (384h54m50.904934287s) [upload=338672]: Next returned nil segment Oct 9 14:26:05.442864387 (384h54m50.90495687s) [upload=338672]: Waiting for error group managing segments... Oct 9 14:26:05.934192787 (384h54m51.39628527s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.967835299 (384h54m51.429927783s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.974753102 (384h54m51.436845586s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.010380625 (384h54m51.472473118s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.021909868 (384h54m51.484002351s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.027655419 (384h54m51.489747902s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.038944626 (384h54m51.501037109s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.095896374 (384h54m51.557988847s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.102013522 (384h54m51.564106006s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.128550885 (384h54m51.590643368s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.134822928 (384h54m51.596915411s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.255075143 (384h54m51.717167626s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345427703 (384h54m51.807520176s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345465595 (384h54m51.807558078s) [seg_pos={0 0}, upload=338672]: Segment reached optimal threshold of 13 pieces. Oct 9 14:26:06.345686406 (384h54m51.807778899s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection Oct 9 14:26:06.345701043 (384h54m51.807793526s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345718196 (384h54m51.807810679s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345741059 (384h54m51.807833542s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345868813 (384h54m51.807961296s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345879122 (384h54m51.807971605s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345921412 (384h54m51.808013886s) [seg_pos={0 0}, upload=338672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:26:06.346037093 (384h54m51.808129577s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:26:06.346044117 (384h54m51.8081366s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:26:06.696762736 (384h54m52.158855219s) [upload=338672]: Done uploading segments... Oct 9 14:26:06.696787322 (384h54m52.158879796s) [upload=338672]: Upload finished. err: <nil> Duration: 11.459887772s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X............X.......X..............X...........X..X..X..X.] duration: 8.996846054s FAIL (2025-10-09 14:26:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA) context deadline exceeded; piecestore: (Node ID: 1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, Piece ID: OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ) context deadline exceeded ======LOGS====== ct 9 14:26:02.931066356 (384h54m48.393158849s) [upload=338672]: Starting upload Oct 9 14:26:02.931150146 (384h54m48.393242629s) [upload=338672]: Done starting upload Oct 9 14:26:02.931263381 (384h54m48.393355864s) [upload=338672]: Uploading segments... Oct 9 14:26:02.931524760 (384h54m48.393617243s) [upload=338672]: Got next segment. Inline: false Oct 9 14:26:02.931579423 (384h54m48.393671906s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:26:02.931586807 (384h54m48.39367928s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:26:03.221262760 (384h54m48.683355253s) [seg_pos={0 0}, upload=338672]: Begin upload segment... Oct 9 14:26:03.221340026 (384h54m48.68343251s) [seg_pos={0 0}, upload=338672]: Done begin upload segment. Oct 9 14:26:03.221378089 (384h54m48.683470572s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.221423807 (384h54m48.68351629s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222245280 (384h54m48.684337773s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222946065 (384h54m48.685038549s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.223607265 (384h54m48.685699748s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.224204682 (384h54m48.686297165s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.225038119 (384h54m48.687130602s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.227192381 (384h54m48.689284864s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.230488067 (384h54m48.69258058s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.231261710 (384h54m48.693354193s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.235934918 (384h54m48.698027401s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.237597444 (384h54m48.699689937s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.240783411 (384h54m48.702875905s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.243963897 (384h54m48.70605638s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.247313737 (384h54m48.70940622s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.252695223 (384h54m48.714787706s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:05.442841804 (384h54m50.904934287s) [upload=338672]: Next returned nil segment Oct 9 14:26:05.442864387 (384h54m50.90495687s) [upload=338672]: Waiting for error group managing segments... Oct 9 14:26:05.934192787 (384h54m51.39628527s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.967835299 (384h54m51.429927783s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.974753102 (384h54m51.436845586s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.010380625 (384h54m51.472473118s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.021909868 (384h54m51.484002351s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.027655419 (384h54m51.489747902s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.038944626 (384h54m51.501037109s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.095896374 (384h54m51.557988847s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.102013522 (384h54m51.564106006s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.128550885 (384h54m51.590643368s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.134822928 (384h54m51.596915411s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.255075143 (384h54m51.717167626s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345427703 (384h54m51.807520176s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345465595 (384h54m51.807558078s) [seg_pos={0 0}, upload=338672]: Segment reached optimal threshold of 13 pieces. Oct 9 14:26:06.345686406 (384h54m51.807778899s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection Oct 9 14:26:06.345701043 (384h54m51.807793526s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345718196 (384h54m51.807810679s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345741059 (384h54m51.807833542s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345868813 (384h54m51.807961296s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345879122 (384h54m51.807971605s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345921412 (384h54m51.808013886s) [seg_pos={0 0}, upload=338672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:26:06.346037093 (384h54m51.808129577s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:26:06.346044117 (384h54m51.8081366s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:26:06.696762736 (384h54m52.158855219s) [upload=338672]: Done uploading segments... Oct 9 14:26:06.696787322 (384h54m52.158879796s) [upload=338672]: Upload finished. err: <nil> Duration: 8.996865121s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X............X.......X..............X...........X..X..X..X] duration: 30.002402246s FAIL (2025-10-09 14:26:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA) context deadline exceeded; piecestore: (Node ID: 1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, Piece ID: OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ) context deadline exceeded ======LOGS====== ct 9 14:26:02.931066356 (384h54m48.393158849s) [upload=338672]: Starting upload Oct 9 14:26:02.931150146 (384h54m48.393242629s) [upload=338672]: Done starting upload Oct 9 14:26:02.931263381 (384h54m48.393355864s) [upload=338672]: Uploading segments... Oct 9 14:26:02.931524760 (384h54m48.393617243s) [upload=338672]: Got next segment. Inline: false Oct 9 14:26:02.931579423 (384h54m48.393671906s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:26:02.931586807 (384h54m48.39367928s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:26:03.221262760 (384h54m48.683355253s) [seg_pos={0 0}, upload=338672]: Begin upload segment... Oct 9 14:26:03.221340026 (384h54m48.68343251s) [seg_pos={0 0}, upload=338672]: Done begin upload segment. Oct 9 14:26:03.221378089 (384h54m48.683470572s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.221423807 (384h54m48.68351629s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222245280 (384h54m48.684337773s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.222946065 (384h54m48.685038549s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.223607265 (384h54m48.685699748s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.224204682 (384h54m48.686297165s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.225038119 (384h54m48.687130602s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.227192381 (384h54m48.689284864s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.230488067 (384h54m48.69258058s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.231261710 (384h54m48.693354193s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.235934918 (384h54m48.698027401s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.237597444 (384h54m48.699689937s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.240783411 (384h54m48.702875905s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.243963897 (384h54m48.70605638s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.247313737 (384h54m48.70940622s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:03.252695223 (384h54m48.714787706s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Uploading piece... Oct 9 14:26:05.442841804 (384h54m50.904934287s) [upload=338672]: Next returned nil segment Oct 9 14:26:05.442864387 (384h54m50.90495687s) [upload=338672]: Waiting for error group managing segments... Oct 9 14:26:05.934192787 (384h54m51.39628527s) [noise=true , address=45.91.116.6:4973 , piece_id=RAUQQ5EPX4YKXBFTZXHYDGXK2OXOJRILYQAK4K42LWDMVT4HUODQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.967835299 (384h54m51.429927783s) [noise=true , address=77.90.31.21:4879 , piece_id=SESQT2KLS7QVGUCL5OTVIXMR5BZWZHWE2KUK6NUWJ7HGWSUSR4IA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:05.974753102 (384h54m51.436845586s) [noise=true , address=45.91.116.11:4119 , piece_id=NFG75AYINYZFSVGYZ3HTDNRDN6N45MIHCNIKWWUDI7XKQJJCEXKQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.010380625 (384h54m51.472473118s) [noise=true , address=145.79.227.16:4909 , piece_id=FYJHIAUD5ZA4VU3YUA4FSAWD74LXTQVRFRER5VX64Y7CXVFGRN3Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.021909868 (384h54m51.484002351s) [noise=true , address=45.91.116.6:4975 , piece_id=HHAF3WK4U2IJMLG6RN7YBRYGEEM4LYJZJ2PMVDNZHGCOG6SZHIWQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.027655419 (384h54m51.489747902s) [noise=true , address=145.79.227.4:4187 , piece_id=AG52LJ46REWDGE25R57BT3NWTEW3PS25PS2HY7EG6SS4XBVMGRLA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.038944626 (384h54m51.501037109s) [noise=true , address=145.79.227.13:4801 , piece_id=SP4BEV4IAZAHS2257SIMTUA4FNMSD2SXTESAZTZ5B6YIAEWGLDNQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.095896374 (384h54m51.557988847s) [noise=true , address=145.79.227.2:4114 , piece_id=QGR37LZ2PAWMAKITUBDWOSLFA3UZPYUAXYNXTFVLDJCZ4UAXO6PA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.102013522 (384h54m51.564106006s) [noise=true , address=145.79.227.15:4875 , piece_id=AZZZ7FRH6YBEWG5BQ5G65RWH24VOLQUXEHCWNE4RQVFHQXPGQ7EA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.128550885 (384h54m51.590643368s) [noise=true , address=45.91.116.4:4937 , piece_id=ZRGVG5DE6DX6667XKU2UD2EAHIZFMZYFMI22VSLR2MTQ3GYCA4QQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.134822928 (384h54m51.596915411s) [noise=true , address=145.79.227.11:4730 , piece_id=3ZBLFXW5A6NHAPK2HKDEPQUOWCC3XZF2JPFHEVA2QUITJ6ZFMHYQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.255075143 (384h54m51.717167626s) [noise=true , address=45.91.116.10:4082 , piece_id=OH6HQVWVZSCR6BTIKQPXWFCTP6QZACUYBPY3MQLSPUTQQTTY4WEQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345427703 (384h54m51.807520176s) [noise=true , address=45.91.116.13:4155 , piece_id=CD2VDFVNZZNSDT7WWHGTSCIQV5245INPJIQNVS3OWTJL2EECOLZA, seg_pos={0 0}, upload=338672]: Done uploading piece. err: <nil> Oct 9 14:26:06.345465595 (384h54m51.807558078s) [seg_pos={0 0}, upload=338672]: Segment reached optimal threshold of 13 pieces. Oct 9 14:26:06.345686406 (384h54m51.807778899s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50762->145.79.227.6:4258: use of closed network connection Oct 9 14:26:06.345701043 (384h54m51.807793526s) [noise=true , address=145.79.227.6:4258 , piece_id=GYBJ45G7TPQ4TOEJVSMD4WL6NJQYTYGWH4LL2QWEGLW5N6CI6T2Q, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345718196 (384h54m51.807810679s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345741059 (384h54m51.807833542s) [noise=true , address=45.91.116.8:4045 , piece_id=2NG7NE4AZMS2LYDBPZ5WON7EZ7DYLPJBKJBD4T4HDSKFN3EQE7TQ, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345868813 (384h54m51.807961296s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 9 14:26:06.345879122 (384h54m51.807971605s) [noise=true , address=45.91.116.4:4939 , piece_id=NRAYJPQ7DWMV6IY47SWCOYVGFC2O5V4TG6XGWYIYOHMQCDXRKY7A, seg_pos={0 0}, upload=338672]: Long tail cancellation due to optimal threshold Oct 9 14:26:06.345921412 (384h54m51.808013886s) [seg_pos={0 0}, upload=338672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:26:06.346037093 (384h54m51.808129577s) [upload=338672]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:26:06.346044117 (384h54m51.8081366s) [upload=338672]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:26:06.696762736 (384h54m52.158855219s) [upload=338672]: Done uploading segments... Oct 9 14:26:06.696787322 (384h54m52.158879796s) [upload=338672]: Upload finished. err: <nil> Duration: 30.002421714s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X............X.......X..............X...........X..X..X..] duration: 18.736037312s FAIL (2025-10-09 14:25:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:23:22.254477251 (384h52m7.716569744s) [upload=338634]: Starting upload Oct 9 14:23:22.254568284 (384h52m7.716660827s) [upload=338634]: Done starting upload Oct 9 14:23:22.254833900 (384h52m7.716926383s) [upload=338634]: Uploading segments... Oct 9 14:23:22.254918741 (384h52m7.717011264s) [upload=338634]: Got next segment. Inline: false Oct 9 14:23:22.254963216 (384h52m7.717055699s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:23:22.254969187 (384h52m7.71706167s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:23:22.511892262 (384h52m7.973984745s) [seg_pos={0 0}, upload=338634]: Begin upload segment... Oct 9 14:23:22.511954950 (384h52m7.974047433s) [seg_pos={0 0}, upload=338634]: Done begin upload segment. Oct 9 14:23:22.512000267 (384h52m7.97409275s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512056073 (384h52m7.974148546s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512972719 (384h52m7.975065212s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513050116 (384h52m7.975142599s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513508689 (384h52m7.975601182s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513660018 (384h52m7.975752531s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514053086 (384h52m7.976145569s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514261583 (384h52m7.976354066s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514568568 (384h52m7.976661051s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514756355 (384h52m7.976848838s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515093738 (384h52m7.977186221s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515267439 (384h52m7.977359922s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515777459 (384h52m7.977869942s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516265720 (384h52m7.978358203s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516768397 (384h52m7.97886088s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.517280292 (384h52m7.979372775s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:24.510528406 (384h52m9.97262089s) [upload=338634]: Next returned nil segment Oct 9 14:23:24.510552722 (384h52m9.972645205s) [upload=338634]: Waiting for error group managing segments... Oct 9 14:23:24.842246015 (384h52m10.304338488s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.853711868 (384h52m10.315804352s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.886859889 (384h52m10.348952362s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.217952377 (384h52m10.68004486s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.262131723 (384h52m10.724224236s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.290069853 (384h52m10.752162356s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.378555108 (384h52m10.840647591s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.402290324 (384h52m10.864382797s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.407141792 (384h52m10.869234265s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408158388 (384h52m10.870250871s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408878299 (384h52m10.870970772s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.416155676 (384h52m10.878248159s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461824400 (384h52m10.923916883s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461848605 (384h52m10.923941078s) [seg_pos={0 0}, upload=338634]: Segment reached optimal threshold of 13 pieces. Oct 9 14:23:25.462056922 (384h52m10.924149405s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462069396 (384h52m10.924161869s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462115303 (384h52m10.924207786s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.462168996 (384h52m10.924261479s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462177903 (384h52m10.924270376s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462221064 (384h52m10.924313548s) [seg_pos={0 0}, upload=338634]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Oct 9 14:23:25.462307570 (384h52m10.924400043s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:23:25.462313551 (384h52m10.924406034s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:23:25.803748643 (384h52m11.265841136s) [upload=338634]: Done uploading segments... Oct 9 14:23:25.803774302 (384h52m11.265866785s) [upload=338634]: Upload finished. err: <nil> Duration: 18.736091967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X............X.......X..............X...........X..X..X.] duration: 9.401815614s FAIL (2025-10-09 14:24:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:23:22.254477251 (384h52m7.716569744s) [upload=338634]: Starting upload Oct 9 14:23:22.254568284 (384h52m7.716660827s) [upload=338634]: Done starting upload Oct 9 14:23:22.254833900 (384h52m7.716926383s) [upload=338634]: Uploading segments... Oct 9 14:23:22.254918741 (384h52m7.717011264s) [upload=338634]: Got next segment. Inline: false Oct 9 14:23:22.254963216 (384h52m7.717055699s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:23:22.254969187 (384h52m7.71706167s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:23:22.511892262 (384h52m7.973984745s) [seg_pos={0 0}, upload=338634]: Begin upload segment... Oct 9 14:23:22.511954950 (384h52m7.974047433s) [seg_pos={0 0}, upload=338634]: Done begin upload segment. Oct 9 14:23:22.512000267 (384h52m7.97409275s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512056073 (384h52m7.974148546s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512972719 (384h52m7.975065212s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513050116 (384h52m7.975142599s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513508689 (384h52m7.975601182s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513660018 (384h52m7.975752531s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514053086 (384h52m7.976145569s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514261583 (384h52m7.976354066s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514568568 (384h52m7.976661051s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514756355 (384h52m7.976848838s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515093738 (384h52m7.977186221s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515267439 (384h52m7.977359922s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515777459 (384h52m7.977869942s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516265720 (384h52m7.978358203s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516768397 (384h52m7.97886088s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.517280292 (384h52m7.979372775s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:24.510528406 (384h52m9.97262089s) [upload=338634]: Next returned nil segment Oct 9 14:23:24.510552722 (384h52m9.972645205s) [upload=338634]: Waiting for error group managing segments... Oct 9 14:23:24.842246015 (384h52m10.304338488s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.853711868 (384h52m10.315804352s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.886859889 (384h52m10.348952362s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.217952377 (384h52m10.68004486s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.262131723 (384h52m10.724224236s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.290069853 (384h52m10.752162356s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.378555108 (384h52m10.840647591s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.402290324 (384h52m10.864382797s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.407141792 (384h52m10.869234265s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408158388 (384h52m10.870250871s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408878299 (384h52m10.870970772s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.416155676 (384h52m10.878248159s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461824400 (384h52m10.923916883s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461848605 (384h52m10.923941078s) [seg_pos={0 0}, upload=338634]: Segment reached optimal threshold of 13 pieces. Oct 9 14:23:25.462056922 (384h52m10.924149405s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462069396 (384h52m10.924161869s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462115303 (384h52m10.924207786s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.462168996 (384h52m10.924261479s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462177903 (384h52m10.924270376s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462221064 (384h52m10.924313548s) [seg_pos={0 0}, upload=338634]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Oct 9 14:23:25.462307570 (384h52m10.924400043s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:23:25.462313551 (384h52m10.924406034s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:23:25.803748643 (384h52m11.265841136s) [upload=338634]: Done uploading segments... Oct 9 14:23:25.803774302 (384h52m11.265866785s) [upload=338634]: Upload finished. err: <nil> Duration: 9.401848478s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....X............X.......X..............X...........X..X..X] duration: 30.002420876s FAIL (2025-10-09 14:23:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 14:23:22.254477251 (384h52m7.716569744s) [upload=338634]: Starting upload Oct 9 14:23:22.254568284 (384h52m7.716660827s) [upload=338634]: Done starting upload Oct 9 14:23:22.254833900 (384h52m7.716926383s) [upload=338634]: Uploading segments... Oct 9 14:23:22.254918741 (384h52m7.717011264s) [upload=338634]: Got next segment. Inline: false Oct 9 14:23:22.254963216 (384h52m7.717055699s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:23:22.254969187 (384h52m7.71706167s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:23:22.511892262 (384h52m7.973984745s) [seg_pos={0 0}, upload=338634]: Begin upload segment... Oct 9 14:23:22.511954950 (384h52m7.974047433s) [seg_pos={0 0}, upload=338634]: Done begin upload segment. Oct 9 14:23:22.512000267 (384h52m7.97409275s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512056073 (384h52m7.974148546s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.512972719 (384h52m7.975065212s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513050116 (384h52m7.975142599s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513508689 (384h52m7.975601182s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.513660018 (384h52m7.975752531s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514053086 (384h52m7.976145569s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514261583 (384h52m7.976354066s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514568568 (384h52m7.976661051s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.514756355 (384h52m7.976848838s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515093738 (384h52m7.977186221s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515267439 (384h52m7.977359922s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.515777459 (384h52m7.977869942s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516265720 (384h52m7.978358203s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.516768397 (384h52m7.97886088s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:22.517280292 (384h52m7.979372775s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Uploading piece... Oct 9 14:23:24.510528406 (384h52m9.97262089s) [upload=338634]: Next returned nil segment Oct 9 14:23:24.510552722 (384h52m9.972645205s) [upload=338634]: Waiting for error group managing segments... Oct 9 14:23:24.842246015 (384h52m10.304338488s) [noise=true , address=45.91.116.9:4262 , piece_id=2VBJO62MD2X632U7D2LMHP5ZILEW2IVR2QQ3325JEEM7UZMV4WOA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.853711868 (384h52m10.315804352s) [noise=true , address=77.90.31.13:4628 , piece_id=7UMZXZB5L6YH4OXH7L4FJ7EK4BOWXEKOYTAK632NRVYXXLVUWRWQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:24.886859889 (384h52m10.348952362s) [noise=true , address=77.90.31.17:4771 , piece_id=SH6M72NDYKGMAD6G5TMQ3LT4GEYDU3XLD37E7IW7W4SLM2OCB74A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.217952377 (384h52m10.68004486s) [noise=true , address=45.91.116.11:4117 , piece_id=7VDQTETFFASDELP3KBW5ZSKZSTSJQMKJL6MKK76MWAYTBAS7HG7A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.262131723 (384h52m10.724224236s) [noise=true , address=77.90.31.18:4807 , piece_id=T5NCLWDD4H637XRAS3N35WUSHMPTRELA6CWG46GBDJOXVGEECTGQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.290069853 (384h52m10.752162356s) [noise=true , address=77.90.31.21:4878 , piece_id=W4Y2PNUBJ73VYIG4MYDQ7K2ZJBBP57DLJ5KMXFIIEJ72RSFBG5TQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.378555108 (384h52m10.840647591s) [noise=true , address=145.79.227.14:4838 , piece_id=VV3JQQTEGFFWSZ2CZRWJL2A7J2ZOGOUMWSVMHMKXPNHBXURF63DQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.402290324 (384h52m10.864382797s) [noise=true , address=145.79.227.5:4221 , piece_id=Y32OYCRL6GAPLPA5AGEJUTYWFSP2ULFGPCX7MVEJZJSGT72TMV2A, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.407141792 (384h52m10.869234265s) [noise=true , address=145.79.227.3:4151 , piece_id=VGDOYZF7ISQR7AQGCOP4BQVXZRU7NVXP4VFIRZZVJNFC3X5PX6VQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408158388 (384h52m10.870250871s) [noise=true , address=145.79.227.5:4223 , piece_id=YAHAJYLODOUVCL2GVB5V6WYNIGTRSCQSP5K3R27C6Z2HXGR4UCYA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.408878299 (384h52m10.870970772s) [noise=true , address=145.79.227.6:4257 , piece_id=XQXAPOG2EYH3GOH5DPDLXPJK76KGIU2HK2LL5KUNO4ENUT3BXVOQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.416155676 (384h52m10.878248159s) [noise=true , address=145.79.227.11:4732 , piece_id=2KDL2VYJYYVIPJ2TT4OUL3KN3F2DGN42QCIERA25P4LXHC6JTT2Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461824400 (384h52m10.923916883s) [noise=true , address=145.79.227.16:4909 , piece_id=UNE62FSQ5ALTDSSECG7G5RF62HWUCLLCOHFOXBNJ474HRV3VFOMQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.461848605 (384h52m10.923941078s) [seg_pos={0 0}, upload=338634]: Segment reached optimal threshold of 13 pieces. Oct 9 14:23:25.462056922 (384h52m10.924149405s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462069396 (384h52m10.924161869s) [noise=true , address=45.91.116.4:4937 , piece_id=7LPLI3CU5I4DVTXN5ZAGK5TK62HMVJIDW53TUTFTEQBH3VEB6MUA, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462115303 (384h52m10.924207786s) [noise=true , address=145.79.227.13:4803 , piece_id=RT4TSQPNUG7WHKLUTRKZ2C4Y7M3SJLIGILWIGMKBMYNZQAMSEM7Q, seg_pos={0 0}, upload=338634]: Done uploading piece. err: <nil> Oct 9 14:23:25.462168996 (384h52m10.924261479s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 14:23:25.462177903 (384h52m10.924270376s) [noise=true , address=77.90.31.16:4733 , piece_id=YA747ADB6FDMUMN7GPCPRSW4BDJBBFODM6GLBVCTWQSBSCNBBHKQ, seg_pos={0 0}, upload=338634]: Long tail cancellation due to optimal threshold Oct 9 14:23:25.462221064 (384h52m10.924313548s) [seg_pos={0 0}, upload=338634]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Oct 9 14:23:25.462307570 (384h52m10.924400043s) [upload=338634]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:23:25.462313551 (384h52m10.924406034s) [upload=338634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:23:25.803748643 (384h52m11.265841136s) [upload=338634]: Done uploading segments... Oct 9 14:23:25.803774302 (384h52m11.265866785s) [upload=338634]: Upload finished. err: <nil> Duration: 30.002459841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X............X.......X..............X...........X..X..] duration: 16.496604641s FAIL (2025-10-09 14:22:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, Piece ID: WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ) context deadline exceeded; piecestore: (Node ID: 12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, Piece ID: R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q) context deadline exceeded ======LOGS====== Oct 9 14:21:20.619775384 (384h50m6.081867867s) [upload=338603]: Starting upload Oct 9 14:21:20.619836902 (384h50m6.081929375s) [upload=338603]: Done starting upload Oct 9 14:21:20.619946470 (384h50m6.082038943s) [upload=338603]: Uploading segments... Oct 9 14:21:20.620012797 (384h50m6.08210528s) [upload=338603]: Got next segment. Inline: false Oct 9 14:21:20.620043856 (384h50m6.082136339s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:21:20.620048254 (384h50m6.082140727s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:21:20.876144773 (384h50m6.338237256s) [seg_pos={0 0}, upload=338603]: Begin upload segment... Oct 9 14:21:20.876207433 (384h50m6.338299916s) [seg_pos={0 0}, upload=338603]: Done begin upload segment. Oct 9 14:21:20.876237671 (384h50m6.338330154s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.876303326 (384h50m6.338395809s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877058192 (384h50m6.339150675s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877456411 (384h50m6.339548894s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877895206 (384h50m6.339987689s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878404116 (384h50m6.340496599s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878841419 (384h50m6.340933902s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.879263152 (384h50m6.341355635s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880385318 (384h50m6.342477801s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880806260 (384h50m6.342898743s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.881873091 (384h50m6.343965564s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.883285541 (384h50m6.345378024s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.885491692 (384h50m6.347584165s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.890474841 (384h50m6.352567314s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.892414484 (384h50m6.354506957s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.893915202 (384h50m6.356007685s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:22.745660937 (384h50m8.20775342s) [upload=338603]: Next returned nil segment Oct 9 14:21:22.745681946 (384h50m8.207774429s) [upload=338603]: Waiting for error group managing segments... Oct 9 14:21:23.167101365 (384h50m8.629193848s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.180686855 (384h50m8.642779328s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.229504925 (384h50m8.691597408s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.266316662 (384h50m8.728409145s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.337986907 (384h50m8.80007938s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.721933450 (384h50m9.184025933s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.724041685 (384h50m9.186134158s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.755070940 (384h50m9.217163423s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773056849 (384h50m9.235149332s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773557773 (384h50m9.235650256s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782680645 (384h50m9.244773128s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782858584 (384h50m9.244951057s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872057137 (384h50m9.33414961s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872077445 (384h50m9.334169918s) [seg_pos={0 0}, upload=338603]: Segment reached optimal threshold of 13 pieces. Oct 9 14:21:23.872224294 (384h50m9.334316787s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872251506 (384h50m9.334343989s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872346507 (384h50m9.33443899s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872354482 (384h50m9.334446965s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872605530 (384h50m9.334698023s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection Oct 9 14:21:23.872619537 (384h50m9.33471202s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872649144 (384h50m9.334741627s) [seg_pos={0 0}, upload=338603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:21:23.872738103 (384h50m9.334830586s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:21:23.872743934 (384h50m9.334836417s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:21:24.193145003 (384h50m9.655237507s) [upload=338603]: Done uploading segments... Oct 9 14:21:24.193173398 (384h50m9.655265871s) [upload=338603]: Upload finished. err: <nil> Duration: 16.496636551s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X............X.......X..............X...........X..X.] duration: 18.281668298s FAIL (2025-10-09 14:22:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, Piece ID: WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ) context deadline exceeded; piecestore: (Node ID: 12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, Piece ID: R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q) context deadline exceeded ======LOGS====== Oct 9 14:21:20.619775384 (384h50m6.081867867s) [upload=338603]: Starting upload Oct 9 14:21:20.619836902 (384h50m6.081929375s) [upload=338603]: Done starting upload Oct 9 14:21:20.619946470 (384h50m6.082038943s) [upload=338603]: Uploading segments... Oct 9 14:21:20.620012797 (384h50m6.08210528s) [upload=338603]: Got next segment. Inline: false Oct 9 14:21:20.620043856 (384h50m6.082136339s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:21:20.620048254 (384h50m6.082140727s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:21:20.876144773 (384h50m6.338237256s) [seg_pos={0 0}, upload=338603]: Begin upload segment... Oct 9 14:21:20.876207433 (384h50m6.338299916s) [seg_pos={0 0}, upload=338603]: Done begin upload segment. Oct 9 14:21:20.876237671 (384h50m6.338330154s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.876303326 (384h50m6.338395809s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877058192 (384h50m6.339150675s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877456411 (384h50m6.339548894s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877895206 (384h50m6.339987689s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878404116 (384h50m6.340496599s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878841419 (384h50m6.340933902s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.879263152 (384h50m6.341355635s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880385318 (384h50m6.342477801s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880806260 (384h50m6.342898743s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.881873091 (384h50m6.343965564s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.883285541 (384h50m6.345378024s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.885491692 (384h50m6.347584165s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.890474841 (384h50m6.352567314s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.892414484 (384h50m6.354506957s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.893915202 (384h50m6.356007685s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:22.745660937 (384h50m8.20775342s) [upload=338603]: Next returned nil segment Oct 9 14:21:22.745681946 (384h50m8.207774429s) [upload=338603]: Waiting for error group managing segments... Oct 9 14:21:23.167101365 (384h50m8.629193848s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.180686855 (384h50m8.642779328s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.229504925 (384h50m8.691597408s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.266316662 (384h50m8.728409145s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.337986907 (384h50m8.80007938s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.721933450 (384h50m9.184025933s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.724041685 (384h50m9.186134158s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.755070940 (384h50m9.217163423s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773056849 (384h50m9.235149332s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773557773 (384h50m9.235650256s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782680645 (384h50m9.244773128s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782858584 (384h50m9.244951057s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872057137 (384h50m9.33414961s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872077445 (384h50m9.334169918s) [seg_pos={0 0}, upload=338603]: Segment reached optimal threshold of 13 pieces. Oct 9 14:21:23.872224294 (384h50m9.334316787s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872251506 (384h50m9.334343989s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872346507 (384h50m9.33443899s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872354482 (384h50m9.334446965s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872605530 (384h50m9.334698023s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection Oct 9 14:21:23.872619537 (384h50m9.33471202s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872649144 (384h50m9.334741627s) [seg_pos={0 0}, upload=338603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:21:23.872738103 (384h50m9.334830586s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:21:23.872743934 (384h50m9.334836417s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:21:24.193145003 (384h50m9.655237507s) [upload=338603]: Done uploading segments... Oct 9 14:21:24.193173398 (384h50m9.655265871s) [upload=338603]: Upload finished. err: <nil> Duration: 18.281704045s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X............X.......X..............X...........X..X] duration: 30.001660553s FAIL (2025-10-09 14:21:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, Piece ID: WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ) context deadline exceeded; piecestore: (Node ID: 12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, Piece ID: R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q) context deadline exceeded ======LOGS====== Oct 9 14:21:20.619775384 (384h50m6.081867867s) [upload=338603]: Starting upload Oct 9 14:21:20.619836902 (384h50m6.081929375s) [upload=338603]: Done starting upload Oct 9 14:21:20.619946470 (384h50m6.082038943s) [upload=338603]: Uploading segments... Oct 9 14:21:20.620012797 (384h50m6.08210528s) [upload=338603]: Got next segment. Inline: false Oct 9 14:21:20.620043856 (384h50m6.082136339s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:21:20.620048254 (384h50m6.082140727s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:21:20.876144773 (384h50m6.338237256s) [seg_pos={0 0}, upload=338603]: Begin upload segment... Oct 9 14:21:20.876207433 (384h50m6.338299916s) [seg_pos={0 0}, upload=338603]: Done begin upload segment. Oct 9 14:21:20.876237671 (384h50m6.338330154s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.876303326 (384h50m6.338395809s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877058192 (384h50m6.339150675s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877456411 (384h50m6.339548894s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.877895206 (384h50m6.339987689s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878404116 (384h50m6.340496599s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.878841419 (384h50m6.340933902s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.879263152 (384h50m6.341355635s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880385318 (384h50m6.342477801s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.880806260 (384h50m6.342898743s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.881873091 (384h50m6.343965564s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.883285541 (384h50m6.345378024s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.885491692 (384h50m6.347584165s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.890474841 (384h50m6.352567314s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.892414484 (384h50m6.354506957s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:20.893915202 (384h50m6.356007685s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Uploading piece... Oct 9 14:21:22.745660937 (384h50m8.20775342s) [upload=338603]: Next returned nil segment Oct 9 14:21:22.745681946 (384h50m8.207774429s) [upload=338603]: Waiting for error group managing segments... Oct 9 14:21:23.167101365 (384h50m8.629193848s) [noise=true , address=77.90.31.13:4628 , piece_id=T2FCTGIPBLTFMVKE2H7COHMNP4RLWBHEVL55DO5Y3E2J3K3TATIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.180686855 (384h50m8.642779328s) [noise=true , address=45.91.116.8:4047 , piece_id=ZXOJISICUAJAN5YBGCMXG6NQNDN5QMGLRQNBPIVFROX4LPQE5SRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.229504925 (384h50m8.691597408s) [noise=true , address=45.91.116.6:4974 , piece_id=3V4AXHFIQKXNUZT4KVGH6WZI5IHRN3NP5GPWUAAAYURQB3MRF2QA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.266316662 (384h50m8.728409145s) [noise=true , address=45.91.116.5:4297 , piece_id=EVO7S5YGDM6OKDAGHVGDR32HOQCSZFRXMRULCZDRMK5N6OY4GBIA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.337986907 (384h50m8.80007938s) [noise=true , address=77.90.31.10:4520 , piece_id=WHSLLA2YC7XHXZOZZDTZM4XJG7WUEWJZQ6L4OGPWUV6LCN3TLYAQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.721933450 (384h50m9.184025933s) [noise=true , address=145.79.227.14:4840 , piece_id=JBRNNBQ42LGCXYNZANULOTVYCLHIGCRRT3YK4VYSPLXOLNYZXLUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.724041685 (384h50m9.186134158s) [noise=true , address=145.79.227.11:4730 , piece_id=BNKEFAYMXKM256H2A4TQPW2IQFU5XJ473SDVI6DZHPDFEPMVWQRQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.755070940 (384h50m9.217163423s) [noise=true , address=145.79.227.6:4257 , piece_id=QL45S7FYV4E7LGN2KHSGMWPHSMSFCVOB4JUVR2IXC4R5DR3GPWVA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773056849 (384h50m9.235149332s) [noise=true , address=145.79.227.12:4767 , piece_id=E3QN6XTN6AYBDOQL5GM4NK7TUO3SWCMEGKIC6AOBCYYPVLYJ3MJQ, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.773557773 (384h50m9.235650256s) [noise=true , address=45.91.116.4:4940 , piece_id=ADZBUHJRU7HL6B5G5LTAZKL6PX4OMRMRAS5KQM57HNWXYO7N3J4A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782680645 (384h50m9.244773128s) [noise=true , address=145.79.227.2:4115 , piece_id=5JMMXSB7VNSWMYB4OF66BQTRJUG2KT4PNW4UA74PCQBEWGCQ4OEA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.782858584 (384h50m9.244951057s) [noise=true , address=145.79.227.7:4293 , piece_id=5SY6JIUFVUU2GQCVCKP6PDOPVTRA7FUJ4XOWZWTN6M2GTKLS6CLA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872057137 (384h50m9.33414961s) [noise=true , address=77.90.31.15:4698 , piece_id=R2IWTB5ZMMRFPLRCZ6D4MTBKIO4ASUWYJYOEPF3ORGBUJMFVAL4Q, seg_pos={0 0}, upload=338603]: Done uploading piece. err: <nil> Oct 9 14:21:23.872077445 (384h50m9.334169918s) [seg_pos={0 0}, upload=338603]: Segment reached optimal threshold of 13 pieces. Oct 9 14:21:23.872224294 (384h50m9.334316787s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872251506 (384h50m9.334343989s) [noise=true , address=45.91.116.9:4263 , piece_id=7CIV3HHPFDLK2ZZT22SYQ4PHP3ATMWHQQI5H5T7APHPTQWOQ7BUA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872346507 (384h50m9.33443899s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Oct 9 14:21:23.872354482 (384h50m9.334446965s) [noise=true , address=77.90.31.13:4626 , piece_id=MVQQBZBMSRHH6QXSICHQF6I2ORLJDF3FBMZPSENQMRKAFEYNJMRA, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872605530 (384h50m9.334698023s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55944->45.91.116.7:4010: use of closed network connection Oct 9 14:21:23.872619537 (384h50m9.33471202s) [noise=true , address=45.91.116.7:4010 , piece_id=MGLQ2IGUY5CLEB7Y5MU7VE7LUDRH5EDS6UD4C3FOUAZ2RTB7PO5A, seg_pos={0 0}, upload=338603]: Long tail cancellation due to optimal threshold Oct 9 14:21:23.872649144 (384h50m9.334741627s) [seg_pos={0 0}, upload=338603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:21:23.872738103 (384h50m9.334830586s) [upload=338603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:21:23.872743934 (384h50m9.334836417s) [upload=338603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:21:24.193145003 (384h50m9.655237507s) [upload=338603]: Done uploading segments... Oct 9 14:21:24.193173398 (384h50m9.655265871s) [upload=338603]: Upload finished. err: <nil> Duration: 30.001676484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X............X.......X..............X...........X..] duration: 27.754388061s FAIL (2025-10-09 14:21:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== g segments... Oct 9 14:19:01.900745461 (384h47m47.362837954s) [upload=338566]: Got next segment. Inline: false Oct 9 14:19:01.900784725 (384h47m47.362877218s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:19:01.900791017 (384h47m47.36288351s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:19:02.199301593 (384h47m47.661394066s) [seg_pos={0 0}, upload=338566]: Begin upload segment... Oct 9 14:19:02.199374913 (384h47m47.661467396s) [seg_pos={0 0}, upload=338566]: Done begin upload segment. Oct 9 14:19:02.199416983 (384h47m47.661509516s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202120752 (384h47m47.664213235s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202838508 (384h47m47.664930991s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.203245383 (384h47m47.665337866s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204655078 (384h47m47.666747601s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204780315 (384h47m47.666872799s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206277857 (384h47m47.66837033s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206387265 (384h47m47.668479748s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.207683003 (384h47m47.669775486s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.208040243 (384h47m47.670132726s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209566590 (384h47m47.671659073s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209620772 (384h47m47.671713256s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211273751 (384h47m47.673366264s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211344886 (384h47m47.673437379s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.212872445 (384h47m47.674964928s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.213582426 (384h47m47.675674909s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:04.279648750 (384h47m49.741741223s) [upload=338566]: Next returned nil segment Oct 9 14:19:04.279668808 (384h47m49.741761291s) [upload=338566]: Waiting for error group managing segments... Oct 9 14:19:04.703817995 (384h47m50.165910478s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.771940394 (384h47m50.234032877s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.772559513 (384h47m50.234651996s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.799199711 (384h47m50.261292235s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.897475158 (384h47m50.359567661s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.922339344 (384h47m50.384431827s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.987149526 (384h47m50.449242009s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.027228595 (384h47m50.489321078s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.113161099 (384h47m50.575253582s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.132007386 (384h47m50.594099869s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.143974223 (384h47m50.606066706s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.149431966 (384h47m50.611524449s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155442711 (384h47m50.617535194s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155476185 (384h47m50.617568668s) [seg_pos={0 0}, upload=338566]: Segment reached optimal threshold of 13 pieces. Oct 9 14:19:05.155685253 (384h47m50.617777746s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection Oct 9 14:19:05.155697497 (384h47m50.61778998s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155894130 (384h47m50.617986614s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection Oct 9 14:19:05.155904411 (384h47m50.617996894s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155980165 (384h47m50.618072648s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Oct 9 14:19:05.155988050 (384h47m50.618080533s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.156007818 (384h47m50.618100301s) [seg_pos={0 0}, upload=338566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:19:05.156094743 (384h47m50.618187226s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:19:05.156100934 (384h47m50.618193417s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:19:05.471703467 (384h47m50.93379596s) [upload=338566]: Done uploading segments... Oct 9 14:19:05.471729917 (384h47m50.93382241s) [upload=338566]: Upload finished. err: <nil> Duration: 27.754592591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X............X.......X..............X...........X.] duration: 16.395767095s FAIL (2025-10-09 14:19:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== g segments... Oct 9 14:19:01.900745461 (384h47m47.362837954s) [upload=338566]: Got next segment. Inline: false Oct 9 14:19:01.900784725 (384h47m47.362877218s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:19:01.900791017 (384h47m47.36288351s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:19:02.199301593 (384h47m47.661394066s) [seg_pos={0 0}, upload=338566]: Begin upload segment... Oct 9 14:19:02.199374913 (384h47m47.661467396s) [seg_pos={0 0}, upload=338566]: Done begin upload segment. Oct 9 14:19:02.199416983 (384h47m47.661509516s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202120752 (384h47m47.664213235s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202838508 (384h47m47.664930991s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.203245383 (384h47m47.665337866s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204655078 (384h47m47.666747601s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204780315 (384h47m47.666872799s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206277857 (384h47m47.66837033s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206387265 (384h47m47.668479748s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.207683003 (384h47m47.669775486s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.208040243 (384h47m47.670132726s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209566590 (384h47m47.671659073s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209620772 (384h47m47.671713256s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211273751 (384h47m47.673366264s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211344886 (384h47m47.673437379s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.212872445 (384h47m47.674964928s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.213582426 (384h47m47.675674909s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:04.279648750 (384h47m49.741741223s) [upload=338566]: Next returned nil segment Oct 9 14:19:04.279668808 (384h47m49.741761291s) [upload=338566]: Waiting for error group managing segments... Oct 9 14:19:04.703817995 (384h47m50.165910478s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.771940394 (384h47m50.234032877s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.772559513 (384h47m50.234651996s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.799199711 (384h47m50.261292235s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.897475158 (384h47m50.359567661s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.922339344 (384h47m50.384431827s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.987149526 (384h47m50.449242009s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.027228595 (384h47m50.489321078s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.113161099 (384h47m50.575253582s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.132007386 (384h47m50.594099869s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.143974223 (384h47m50.606066706s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.149431966 (384h47m50.611524449s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155442711 (384h47m50.617535194s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155476185 (384h47m50.617568668s) [seg_pos={0 0}, upload=338566]: Segment reached optimal threshold of 13 pieces. Oct 9 14:19:05.155685253 (384h47m50.617777746s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection Oct 9 14:19:05.155697497 (384h47m50.61778998s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155894130 (384h47m50.617986614s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection Oct 9 14:19:05.155904411 (384h47m50.617996894s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155980165 (384h47m50.618072648s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Oct 9 14:19:05.155988050 (384h47m50.618080533s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.156007818 (384h47m50.618100301s) [seg_pos={0 0}, upload=338566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:19:05.156094743 (384h47m50.618187226s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:19:05.156100934 (384h47m50.618193417s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:19:05.471703467 (384h47m50.93379596s) [upload=338566]: Done uploading segments... Oct 9 14:19:05.471729917 (384h47m50.93382241s) [upload=338566]: Upload finished. err: <nil> Duration: 16.395798485s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X............X.......X..............X...........X] duration: 30.002310459s FAIL (2025-10-09 14:19:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== g segments... Oct 9 14:19:01.900745461 (384h47m47.362837954s) [upload=338566]: Got next segment. Inline: false Oct 9 14:19:01.900784725 (384h47m47.362877218s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:19:01.900791017 (384h47m47.36288351s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:19:02.199301593 (384h47m47.661394066s) [seg_pos={0 0}, upload=338566]: Begin upload segment... Oct 9 14:19:02.199374913 (384h47m47.661467396s) [seg_pos={0 0}, upload=338566]: Done begin upload segment. Oct 9 14:19:02.199416983 (384h47m47.661509516s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202120752 (384h47m47.664213235s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.202838508 (384h47m47.664930991s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.203245383 (384h47m47.665337866s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204655078 (384h47m47.666747601s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.204780315 (384h47m47.666872799s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206277857 (384h47m47.66837033s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.206387265 (384h47m47.668479748s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.207683003 (384h47m47.669775486s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.208040243 (384h47m47.670132726s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209566590 (384h47m47.671659073s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.209620772 (384h47m47.671713256s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211273751 (384h47m47.673366264s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.211344886 (384h47m47.673437379s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.212872445 (384h47m47.674964928s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:02.213582426 (384h47m47.675674909s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Uploading piece... Oct 9 14:19:04.279648750 (384h47m49.741741223s) [upload=338566]: Next returned nil segment Oct 9 14:19:04.279668808 (384h47m49.741761291s) [upload=338566]: Waiting for error group managing segments... Oct 9 14:19:04.703817995 (384h47m50.165910478s) [noise=true , address=77.90.31.18:4805 , piece_id=D6PKVTSWIZKOAUGGXMEEPHTHCIXFISATCFOXWAYSTXPUDHPGCJNQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.771940394 (384h47m50.234032877s) [noise=true , address=45.91.116.13:4154 , piece_id=YFDSQ3SPFSBWKO43JSI2GULZPX3KNUDSGEIHNCD6ETNOZDNZRJJQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.772559513 (384h47m50.234651996s) [noise=true , address=45.91.116.11:4118 , piece_id=L2E2VW2UNN5P4LMO32S5JKLGXRKRHF45TCXG46TPV46QQCUVZOZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.799199711 (384h47m50.261292235s) [noise=true , address=45.91.116.5:4299 , piece_id=TL2JL42T2BYOSUN44F3RT234JHDNV7YIUBEW6BHHWOXGZSMFLQXQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.897475158 (384h47m50.359567661s) [noise=true , address=45.91.116.4:4937 , piece_id=MMDBY7TXXAEYEKYMOS34T2O62LR63K53SPHLYVLAMNGYIKRMHFPA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.922339344 (384h47m50.384431827s) [noise=true , address=77.90.31.17:4770 , piece_id=5WRLLXSCJY6TCLJ2SJYVFXARP3BA6XRMNCXW2FCC3XKGN33RBVYQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:04.987149526 (384h47m50.449242009s) [noise=true , address=77.90.31.11:4553 , piece_id=EZXD6UIGWRWST7TJBP4MCM6ONLOOCQBOXO3YRCJCJIJQMPGUTFIA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.027228595 (384h47m50.489321078s) [noise=true , address=145.79.227.5:4222 , piece_id=SB35G3ZFLPTDHKG5SDH5W7G35LXVBO3YVGNVKXJPCFF4DAO3ZFLA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.113161099 (384h47m50.575253582s) [noise=true , address=145.79.227.2:4115 , piece_id=4X5KIWVJWRIEEDNQOFAUTPZ2T6IQFNQ64KU35OULHKCZ4BBB57XA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.132007386 (384h47m50.594099869s) [noise=true , address=145.79.227.11:4731 , piece_id=BACYCAQPSL42GZ63E7WK4R5SJEHDUWFQE7YYJZ2N6C5NDUJK3VZQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.143974223 (384h47m50.606066706s) [noise=true , address=145.79.227.14:4837 , piece_id=ZV7UNIMHRBWI7VSSUO6FROPN7EXQ56XI3TOBGYOOKKF24D4P7IQA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.149431966 (384h47m50.611524449s) [noise=true , address=45.91.116.10:4083 , piece_id=DY53AHLPOBQJCLC7JHMOMTGUJAX3ASUHTDOUUF5GJUYRQFPERH5Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155442711 (384h47m50.617535194s) [noise=true , address=145.79.227.14:4838 , piece_id=D7NH4JMGXT5JHC66RWSNPRZQ2K5SUBFSSBR64CL3NN3CFNJ4RANQ, seg_pos={0 0}, upload=338566]: Done uploading piece. err: <nil> Oct 9 14:19:05.155476185 (384h47m50.617568668s) [seg_pos={0 0}, upload=338566]: Segment reached optimal threshold of 13 pieces. Oct 9 14:19:05.155685253 (384h47m50.617777746s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42254->77.90.31.10:4520: use of closed network connection Oct 9 14:19:05.155697497 (384h47m50.61778998s) [noise=true , address=77.90.31.10:4520 , piece_id=CFDK4HDG3TA44Z6Z6ETY63F2XFOLQ7IHHD2KIFQBBNTIDENUSM4A, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155894130 (384h47m50.617986614s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41112->45.91.116.5:4298: use of closed network connection Oct 9 14:19:05.155904411 (384h47m50.617996894s) [noise=true , address=45.91.116.5:4298 , piece_id=VDALJCPCJOOEYNFKGB3ICJJAQWOTV35IRAD4TAPT4FE3GRF2XL3Q, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.155980165 (384h47m50.618072648s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Oct 9 14:19:05.155988050 (384h47m50.618080533s) [noise=true , address=145.79.227.7:4295 , piece_id=VBNVNC6RMT4QVTM22KYNJ5ODU4ILSOIBMIJYZ4CDZPVQ2DSM5MXA, seg_pos={0 0}, upload=338566]: Long tail cancellation due to optimal threshold Oct 9 14:19:05.156007818 (384h47m50.618100301s) [seg_pos={0 0}, upload=338566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:19:05.156094743 (384h47m50.618187226s) [upload=338566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:19:05.156100934 (384h47m50.618193417s) [upload=338566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:19:05.471703467 (384h47m50.93379596s) [upload=338566]: Done uploading segments... Oct 9 14:19:05.471729917 (384h47m50.93382241s) [upload=338566]: Upload finished. err: <nil> Duration: 30.002341216s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X............X.......X..............X...........] duration: 13.316780216s FAIL (2025-10-09 14:18:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 13.316817646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X............X.......X..............X..........] duration: 10.487764443s FAIL (2025-10-09 14:17:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 10.487790561s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X............X.......X..............X.........] duration: 20.411170539s FAIL (2025-10-09 14:17:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 20.411204202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X............X.......X..............X........] duration: 26.485228503s FAIL (2025-10-09 14:16:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 26.485260615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X............X.......X..............X.......] duration: 25.539082245s FAIL (2025-10-09 14:16:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 25.539115288s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X............X.......X..............X......] duration: 12.963050013s FAIL (2025-10-09 14:15:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 12.963080331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.................X............X.......X..............X.....] duration: 27.313445745s FAIL (2025-10-09 14:14:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 27.313481183s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.................X............X.......X..............X....] duration: 16.851006467s FAIL (2025-10-09 14:13:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 16.851038839s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.X.................X............X.......X..............X...] duration: 15.388723414s FAIL (2025-10-09 14:12:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 15.388755495s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX.X.................X............X.......X..............X..] duration: 17.470856782s FAIL (2025-10-09 14:12:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 17.470886589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.XX.X.................X............X.......X..............X.] duration: 15.837235406s FAIL (2025-10-09 14:11:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 15.837266435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..XX.X.................X............X.......X..............X] duration: 30.001991101s FAIL (2025-10-09 14:10:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ) context deadline exceeded; piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ) context deadline exceeded ======LOGS====== Oct 9 14:10:25.203977538 (384h39m10.666070021s) [upload=338437]: Starting upload Oct 9 14:10:25.204100411 (384h39m10.666192904s) [upload=338437]: Done starting upload Oct 9 14:10:25.204323186 (384h39m10.666415679s) [upload=338437]: Uploading segments... Oct 9 14:10:25.204423567 (384h39m10.66651605s) [upload=338437]: Got next segment. Inline: false Oct 9 14:10:25.204531332 (384h39m10.666623825s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 14:10:25.204538886 (384h39m10.666631369s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 14:10:25.462302431 (384h39m10.924394924s) [seg_pos={0 0}, upload=338437]: Begin upload segment... Oct 9 14:10:25.462373987 (384h39m10.92446647s) [seg_pos={0 0}, upload=338437]: Done begin upload segment. Oct 9 14:10:25.462402341 (384h39m10.924494824s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462419204 (384h39m10.924511687s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462469339 (384h39m10.924561812s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.462900349 (384h39m10.924992832s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463282126 (384h39m10.925374649s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.463658734 (384h39m10.925751207s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464003520 (384h39m10.926096003s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.464340893 (384h39m10.926433376s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465006300 (384h39m10.927098793s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.465497165 (384h39m10.927589648s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.466477020 (384h39m10.928569503s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.469909266 (384h39m10.932001749s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.474356284 (384h39m10.936448757s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.476059528 (384h39m10.938152041s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.482690203 (384h39m10.944782686s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:25.484796224 (384h39m10.946888707s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Uploading piece... Oct 9 14:10:27.413054091 (384h39m12.875146574s) [upload=338437]: Next returned nil segment Oct 9 14:10:27.413076304 (384h39m12.875168777s) [upload=338437]: Waiting for error group managing segments... Oct 9 14:10:27.834028826 (384h39m13.296121309s) [noise=true , address=45.91.116.8:4048 , piece_id=NRJLAQVI3ELEBSXMOO2PHISHJ6LM7QS43RU5VKGZBENHLHJOTQCQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:27.963853277 (384h39m13.42594576s) [noise=true , address=45.91.116.11:4118 , piece_id=MBHSZLQ23DOAO7L3XMROT6QWHXFK3NQUJWYNTAQD4PZJHYNNXOJA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.037947348 (384h39m13.500039831s) [noise=true , address=45.91.116.7:4010 , piece_id=UQH3TUCEBO2ZKCWPQ3BNUK4YZJLDBHWFRKK6JHBHZWB6LNTEHNOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.172187649 (384h39m13.634280132s) [noise=true , address=145.79.227.16:4912 , piece_id=LI54YWPWKXOAQVBY7RPG3TPHWDERJJFYVLMTMWP5YY7QCXOBBX7Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.194571754 (384h39m13.656664347s) [noise=true , address=77.90.31.14:4662 , piece_id=EFNBRQHXBV4PHQ5HDLMJFNUXRFCDG7QTMVVHSTZHVWPUFLDEC3RQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.219867141 (384h39m13.681959624s) [noise=true , address=145.79.227.18:4006 , piece_id=3JFB6MDC6OGBTSI6TBA2V65BEGYPCYQQENDIUX2LVLBUK3CMSVXA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.228338112 (384h39m13.690430585s) [noise=true , address=145.79.227.3:4150 , piece_id=LIQIV6FS4XAKFEZHPAK2NOBJWUTZVQ7AVFUTTGWLSC4KBQQKZPEQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.234561613 (384h39m13.696654096s) [noise=true , address=145.79.227.6:4259 , piece_id=RV6KHBQ3AFKDNXV6CBFAB5NPMAPPBXAS372HDHEH4ZOT62FX6EZA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251197090 (384h39m13.713289563s) [noise=true , address=145.79.227.7:4293 , piece_id=XI4IGZOJ2DNNKB7RGVTFL5J5EP45NRGNQNWQA4OIDJ3W3BOLMY6Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.251705829 (384h39m13.713798322s) [noise=true , address=145.79.227.17:4970 , piece_id=CFPJJU7AGAJZD65Y4FW6CQUSTKREXZKXJHYNKIGAQBFSHFAIG36A, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.270112428 (384h39m13.732204912s) [noise=true , address=145.79.227.4:4186 , piece_id=GSXJC2M3SJ5FTWLRAWLZPJIYXYCLIMFIWKHQAHSAYRHJDAWJRQOQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.282531487 (384h39m13.74462397s) [noise=true , address=145.79.227.15:4875 , piece_id=UMYCVUIPGLBUXA4WBVZJ3MJ3QI3ASR5TMMBES6SNNCHRQ4G5WO3Q, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398727298 (384h39m13.860819791s) [noise=true , address=45.91.116.13:4153 , piece_id=UE4NSHY7QBMQUODTORQYLE3365QRZKE7X4K4ZTZQEQ55TLKD3ICQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: <nil> Oct 9 14:10:28.398762134 (384h39m13.860854627s) [seg_pos={0 0}, upload=338437]: Segment reached optimal threshold of 13 pieces. Oct 9 14:10:28.398910035 (384h39m13.861002518s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.398918442 (384h39m13.861010925s) [noise=true , address=77.90.31.21:4877 , piece_id=I3S5E7JR2LYOP2FVNQBA2PQVHGZWXQRMI25YDJU42MKOADUQFYWA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399001760 (384h39m13.861094243s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399009184 (384h39m13.861101667s) [noise=true , address=45.91.116.4:4937 , piece_id=TUHRQFAGNO7E43DZEPT3PESCQ4ETQIUCDLPYNRDAIXQCDUORT5HQ, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399082173 (384h39m13.861174656s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Oct 9 14:10:28.399090229 (384h39m13.861182722s) [noise=true , address=145.79.227.14:4838 , piece_id=OV57AA75EJM772Z34JPCI6BYSPQAUJMCHKLMAPVOHT3BHENS6RCA, seg_pos={0 0}, upload=338437]: Long tail cancellation due to optimal threshold Oct 9 14:10:28.399106370 (384h39m13.861198853s) [seg_pos={0 0}, upload=338437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 14:10:28.399197363 (384h39m13.861289846s) [upload=338437]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 14:10:28.399203585 (384h39m13.861296078s) [upload=338437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 14:10:28.746453851 (384h39m14.208546344s) [upload=338437]: Done uploading segments... Oct 9 14:10:28.746481965 (384h39m14.208574448s) [upload=338437]: Upload finished. err: <nil> Duration: 30.002017281s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..XX.X.................X............X.......X..............] duration: 15.720045406s FAIL (2025-10-09 14:09:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 15.720069582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X..XX.X.................X............X.......X.............] duration: 7.98376986s FAIL (2025-10-09 14:08:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 7.983803304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X..XX.X.................X............X.......X............] duration: 9.800252926s FAIL (2025-10-09 14:07:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 9.800285667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.X..XX.X.................X............X.......X...........] duration: 9.143182222s FAIL (2025-10-09 14:06:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 9.143217882s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XX.X..XX.X.................X............X.......X..........] duration: 24.017605629s FAIL (2025-10-09 14:06:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 24.017639985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XX.X..XX.X.................X............X.......X.........] duration: 14.622680713s FAIL (2025-10-09 14:05:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 14.622748602s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XX.X..XX.X.................X............X.......X........] duration: 8.300791989s FAIL (2025-10-09 14:03:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 8.300851461s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X...XX.X..XX.X.................X............X.......X.......] duration: 14.757827403s FAIL (2025-10-09 14:03:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 14.757860356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XX...XX.X..XX.X.................X............X.......X......] duration: 8.159778394s FAIL (2025-10-09 14:02:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 8.159874859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.XX...XX.X..XX.X.................X............X.......X.....] duration: 22.381392539s FAIL (2025-10-09 14:02:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 22.381456933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XX...XX.X..XX.X.................X............X.......X....] duration: 19.83937965s FAIL (2025-10-09 14:01:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 13:58:26.187058220 (384h27m11.649150703s) [upload=338260]: Starting upload Oct 9 13:58:26.187153511 (384h27m11.649245984s) [upload=338260]: Done starting upload Oct 9 13:58:26.187228655 (384h27m11.649321138s) [upload=338260]: Uploading segments... Oct 9 13:58:26.187450367 (384h27m11.64954284s) [upload=338260]: Got next segment. Inline: false Oct 9 13:58:26.187487858 (384h27m11.649580341s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 13:58:26.187493729 (384h27m11.649586212s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 13:58:26.448881115 (384h27m11.910973598s) [seg_pos={0 0}, upload=338260]: Begin upload segment... Oct 9 13:58:26.449022083 (384h27m11.911114576s) [seg_pos={0 0}, upload=338260]: Done begin upload segment. Oct 9 13:58:26.449035810 (384h27m11.911128303s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.449057350 (384h27m11.911149834s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450021817 (384h27m11.9121143s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450407071 (384h27m11.912499544s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.450739735 (384h27m11.912832208s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451078840 (384h27m11.913171323s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451431561 (384h27m11.913524034s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.451732615 (384h27m11.913825098s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452038357 (384h27m11.91413083s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452374327 (384h27m11.9144668s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.452783325 (384h27m11.914875798s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.453132951 (384h27m11.915225434s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.454164596 (384h27m11.916257079s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.455207101 (384h27m11.917299584s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.456741943 (384h27m11.918834427s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:26.458021871 (384h27m11.920114344s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Uploading piece... Oct 9 13:58:28.398738278 (384h27m13.860830751s) [upload=338260]: Next returned nil segment Oct 9 13:58:28.398795166 (384h27m13.860887659s) [upload=338260]: Waiting for error group managing segments... Oct 9 13:58:28.724780852 (384h27m14.186873345s) [noise=true , address=45.91.116.8:4048 , piece_id=VOL4DXKPLSJ25JLOQFBNFKCRMXMRGQ42KEZMWYBKGS6IETPBPGOA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.735406665 (384h27m14.197499148s) [noise=true , address=45.91.116.5:4299 , piece_id=EOPAWGRETNQ6DYAMEOUE3GT5UGOXML25IWVVWLCQ5A3HWZ6G4ACQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.896265133 (384h27m14.358357606s) [noise=true , address=77.90.31.12:4592 , piece_id=D4RQDCNGNE4YFBZJOV7MDSJUSX2AQV44AMXW765X3W7JMZWVCZMQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:28.931714375 (384h27m14.393806858s) [noise=true , address=77.90.31.15:4697 , piece_id=CG7JRYBEDHJ5XE5R5MXIP4XIIWL7FBZVCGASR7NJ7PRKD5WFX7HA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.082114539 (384h27m14.544207022s) [noise=true , address=77.90.31.16:4734 , piece_id=FJVYVL5PQPHCA7I72MUMXUUL2LQRFE3MZAXCVGUB7MNMOVVAO7MQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.190158026 (384h27m14.652250499s) [noise=true , address=45.91.116.13:4154 , piece_id=DMTHMYIQ7KSTRBHYPCNJKAL7YCKVOPWOM5O5B46IV3INJJYLPEPQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.255136298 (384h27m14.717228782s) [noise=true , address=77.90.31.13:4626 , piece_id=DLOWJEMK276QEFAJMHUHBINEL7UQZ3SBTLO3OEMV7XQ5D7BJSE5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.277942516 (384h27m14.740034999s) [noise=true , address=145.79.227.6:4257 , piece_id=O62V7NEIKWMPRZD6R2Q7WVYCVYCTZ4KC5HGFJZAQMQUJPCFFFASQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.278992275 (384h27m14.741084748s) [noise=true , address=145.79.227.15:4875 , piece_id=UEBOOJYS5ZTYBRX2MRUXP3YIN7FQASAOC6ILSFD5QXQ3OOC47WKQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.296662192 (384h27m14.758754685s) [noise=true , address=145.79.227.11:4729 , piece_id=PW4OZXPZTLZZZLQ4GOTNXZUF4YDZRYRKCWJIUT4BOEZAM4V2LIEQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.301085695 (384h27m14.763178178s) [noise=true , address=145.79.227.2:4115 , piece_id=HZNE7A5ICJJ4FV6KR7AIRV6DGLBFTMMBSFVUBK42GJADTZA72EVA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.302539283 (384h27m14.764631766s) [noise=true , address=145.79.227.12:4767 , piece_id=G2BTGTVZDU3O7FEELLHHCECVX4JWVTEHKUPP3QC3KBFA2EZGHX5Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316628261 (384h27m14.778720744s) [noise=true , address=145.79.227.3:4151 , piece_id=JQ5IBJMB6SRZ6U2LFV65PFJJWJMYNX3XDXCFVHGXZ2HILQIBMQXQ, seg_pos={0 0}, upload=338260]: Done uploading piece. err: <nil> Oct 9 13:58:29.316657207 (384h27m14.77874969s) [seg_pos={0 0}, upload=338260]: Segment reached optimal threshold of 13 pieces. Oct 9 13:58:29.316800199 (384h27m14.778892692s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316810589 (384h27m14.778903072s) [noise=true , address=145.79.227.2:4114 , piece_id=F6KQSFQHKHNN45J75ZYZU3PJL6O7WYW2IN2RIEMSM7TGEYL4OYZA, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.316930748 (384h27m14.779023231s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.316939524 (384h27m14.779032007s) [noise=true , address=145.79.227.5:4222 , piece_id=5J6QLCW46U5Y4VBZDKHX54VTM5RLJCE2UFGES3MSWNIGVFTMJ34Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317020589 (384h27m14.779113072s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Oct 9 13:58:29.317029185 (384h27m14.779121668s) [noise=true , address=145.79.227.18:4005 , piece_id=HBCHAB7RTQKQYTHZWTUOAO4EWDQCENP4IFTQVMWU35XDJLOP3A4Q, seg_pos={0 0}, upload=338260]: Long tail cancellation due to optimal threshold Oct 9 13:58:29.317049484 (384h27m14.779141967s) [seg_pos={0 0}, upload=338260]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 13:58:29.317138994 (384h27m14.779231477s) [upload=338260]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 13:58:29.317147179 (384h27m14.779239663s) [upload=338260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 13:58:29.756813967 (384h27m15.21890646s) [upload=338260]: Done uploading segments... Oct 9 13:58:29.756837882 (384h27m15.218930365s) [upload=338260]: Upload finished. err: <nil> Duration: 19.839411782s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)