Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.452034568s FAIL (2025-07-05 17:33:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.452057031s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.728145688s FAIL (2025-07-05 17:33:43 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.728170314s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.724060082s FAIL (2025-07-05 17:33:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.724091492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.69515741s FAIL (2025-07-05 17:32:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.695181055s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.472015577s FAIL (2025-07-05 17:32:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.472035675s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.7285329s FAIL (2025-07-05 17:31:50 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.72855891s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.474040046s FAIL (2025-07-05 17:31:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.47405834s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.622754691s FAIL (2025-07-05 17:31:14 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.622782887s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.45388569s FAIL (2025-07-05 17:30:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.453905208s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.655709726s FAIL (2025-07-05 17:30:17 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.655744794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.769483312s FAIL (2025-07-05 17:29:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.769513221s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.463884301s FAIL (2025-07-05 17:29:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.46390556s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.81677325s FAIL (2025-07-05 17:29:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.816803247s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.68920196s FAIL (2025-07-05 17:29:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.689216076s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001437684s FAIL (2025-07-05 17:29:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001461829s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.765702564s FAIL (2025-07-05 17:29:00 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 5.765725687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002377673s FAIL (2025-07-05 17:28:38 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ent. successful: 13 optimal: 13 errs: <nil> Jul 5 17:27:16.571809944 (296h16m51.342451279s) [upload=236932]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:27:16.571815556 (296h16m51.342456891s) [upload=236932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:27:16.863746763 (296h16m51.634388098s) [upload=236932]: Done uploading segments... Jul 5 17:27:16.863764737 (296h16m51.634406072s) [upload=236932]: Upload finished. err: <nil> Jul 5 17:28:09.347078064 (296h17m44.117719429s) [upload=236945]: Starting upload Jul 5 17:28:09.347155812 (296h17m44.117797177s) [upload=236945]: Done starting upload Jul 5 17:28:09.347274910 (296h17m44.117916235s) [upload=236945]: Uploading segments... Jul 5 17:28:09.347379050 (296h17m44.118020375s) [upload=236945]: Got next segment. Inline: false Jul 5 17:28:09.347407714 (296h17m44.118049049s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 17:28:09.347412954 (296h17m44.118054289s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 17:28:09.588206709 (296h17m44.358848034s) [seg_pos={0 0}, upload=236945]: Begin upload segment... Jul 5 17:28:09.588274739 (296h17m44.358916074s) [seg_pos={0 0}, upload=236945]: Done begin upload segment. Jul 5 17:28:09.588323944 (296h17m44.358965279s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588379069 (296h17m44.359020404s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588734989 (296h17m44.359376314s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.588935924 (296h17m44.359577259s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589011298 (296h17m44.359652633s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.589270053 (296h17m44.359911389s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590002835 (296h17m44.36064416s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590746688 (296h17m44.361388043s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.590753250 (296h17m44.361394585s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591032105 (296h17m44.36167344s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591093732 (296h17m44.361735067s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591452879 (296h17m44.362094215s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591644045 (296h17m44.36228538s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.591713738 (296h17m44.362355063s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592268069 (296h17m44.362909394s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:09.592567602 (296h17m44.363208937s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Uploading piece... Jul 5 17:28:11.227912581 (296h17m45.998553916s) [upload=236945]: Next returned nil segment Jul 5 17:28:11.227925805 (296h17m45.99856714s) [upload=236945]: Waiting for error group managing segments... Jul 5 17:28:11.529771340 (296h17m46.300412675s) [noise=true , address=104.36.82.9:4005 , piece_id=4ROO3D7X5OS2NB5P4LNBO52P3JVDKYNDJ7TGOJKV3ZEXFT246JFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.537274264 (296h17m46.307915599s) [noise=true , address=104.36.82.12:4151 , piece_id=BLFWHYFODXCBQ2SG4EYTUYI7OSL4IJ5TQKQYYSK2QP3JA5W6WLPQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.543770743 (296h17m46.314412088s) [noise=true , address=104.36.82.7:4912 , piece_id=K324RUVW433UI7AVTICBISWFSKBXUHYXEKKH3IYPLKGGMDY3YQRQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.562362180 (296h17m46.333003515s) [noise=true , address=104.36.82.4:4803 , piece_id=NMBPTNHVMDM5YQGKAFUB5WBMUKAXOOI5MZXS7JZREH6FSJT47APQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.599523163 (296h17m46.370164498s) [noise=true , address=104.36.82.10:4078 , piece_id=MZI3RVLAH7S75N7V4XMHUIJWKMXJGGL67PMBGKVKTOOMP5RBT4RQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.639344807 (296h17m46.409986142s) [noise=true , address=104.36.82.7:4911 , piece_id=DOIBVHBW2AWJSTU424AR3DUST7UN5SZF46J354HRGE5YPBJJE6YQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.657182702 (296h17m46.427824037s) [noise=true , address=104.36.82.8:4972 , piece_id=XYB5OPHSXXBWWCB4KR44YD7NETSHCRWVR7AKR3OJLW22FRY6UYFQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.670611894 (296h17m46.441253229s) [noise=true , address=104.36.82.16:4294 , piece_id=CND4SQ3EQZLCLU2X7NRAJCEJKQKGIZ2DTYRP3EJ7WHT6VZP7INCQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:11.694057187 (296h17m46.464698522s) [noise=true , address=104.36.82.6:4875 , piece_id=QGTENLGFP37WXWNGNCCNENEYC4SSMDLS7ULV6KYRJPGPVECG7QBQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:29.767976961 (296h18m4.538618326s) [noise=true , address=45.91.116.9:4263 , piece_id=IOKGELGHJL5D3UG4JU2G5DKQT6QGASJVINCHZ5ES52E5SQBCARWA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:30.958706550 (296h18m5.729347885s) [noise=true , address=45.91.116.11:4117 , piece_id=UZMQT5OX63U3V2NFB5G24TK72OUNELOQXJMFK2QTSSRAHQILSKTA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:34.136679007 (296h18m8.907320332s) [noise=true , address=45.91.116.7:4010 , piece_id=IRLKDHEG6YR457FHUI756H2HDI36BCOLBBGRNXMMLSY4WUQUQ3CQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719142580 (296h18m11.489783905s) [noise=true , address=77.90.31.11:4554 , piece_id=56XVAHSDXTEGBNFWUROD6S652IXZAAAQLVFDSZFDXGY7E7WQOL3Q, seg_pos={0 0}, upload=236945]: Done uploading piece. err: <nil> Jul 5 17:28:36.719168730 (296h18m11.489810085s) [seg_pos={0 0}, upload=236945]: Segment reached optimal threshold of 13 pieces. Jul 5 17:28:36.719447222 (296h18m11.490088547s) [noise=true , address=45.91.116.5:4299 , piece_id=53NIS5Y5LNHPHGKYFDNULTIOQUEXXMRWUWV4DOAFKOG54CJYA6OQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56806->45.91.116.5:4299: use of closed network connection Jul 5 17:28:36.719487790 (296h18m11.490129126s) [noise=true , address=77.90.31.12:4590 , piece_id=CV65OM7ZVPVGGAX4LG425RXLJCJH2JQFLCJFTVH7NZEEYHK7UHAQ, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 5 17:28:36.719529169 (296h18m11.490170514s) [noise=true , address=77.90.31.16:4734 , piece_id=4U2342LJM6ETBICXBA6TTJUEV2JNKJZZ76Z2TPECZ52D7GKGLFXA, seg_pos={0 0}, upload=236945]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45598->77.90.31.16:4734: use of closed network connection Jul 5 17:28:36.719580027 (296h18m11.490221362s) [seg_pos={0 0}, upload=236945]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 5 17:28:36.719668395 (296h18m11.49030973s) [upload=236945]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 17:28:36.719674027 (296h18m11.490315362s) [upload=236945]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 17:28:37.004680607 (296h18m11.775321942s) [upload=236945]: Done uploading segments... Jul 5 17:28:37.004698051 (296h18m11.775339386s) [upload=236945]: Upload finished. err: <nil> Duration: 30.002401569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.651862335s FAIL (2025-07-05 16:03:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.651886862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.750189107s FAIL (2025-07-05 16:03:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.750245054s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.652573443s FAIL (2025-07-05 16:02:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.652601856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.674687366s FAIL (2025-07-05 16:01:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.674718795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.619308984s FAIL (2025-07-05 16:00:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.619342058s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.826228742s FAIL (2025-07-05 15:59:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.826254844s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.66965111s FAIL (2025-07-05 15:59:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.669675919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.658738896s FAIL (2025-07-05 15:58:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.658770787s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.007556315s FAIL (2025-07-05 15:58:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.007584872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.649043921s FAIL (2025-07-05 15:57:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.649078488s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.641849845s FAIL (2025-07-05 15:56:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.641877419s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.759514635s FAIL (2025-07-05 15:55:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.759540404s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.832824317s FAIL (2025-07-05 15:54:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.832855075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.690291337s FAIL (2025-07-05 15:53:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.690319702s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.836736884s FAIL (2025-07-05 15:52:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.83676515s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.62160611s FAIL (2025-07-05 15:51:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.621631859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.77727649s FAIL (2025-07-05 15:50:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.777299884s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.79520004s FAIL (2025-07-05 15:49:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.795224969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.781324267s FAIL (2025-07-05 15:49:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.781351768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.725059767s FAIL (2025-07-05 15:48:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.725084033s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.600264577s FAIL (2025-07-05 15:47:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.600292851s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.926461417s FAIL (2025-07-05 15:46:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.926487066s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.624486369s FAIL (2025-07-05 15:45:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.624521886s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.70554898s FAIL (2025-07-05 15:44:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.705575371s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.884479712s FAIL (2025-07-05 15:43:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.884506764s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.68693034s FAIL (2025-07-05 15:43:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.686953465s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.669857531s FAIL (2025-07-05 15:42:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.669928747s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.635347621s FAIL (2025-07-05 15:42:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.635388588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.655894045s FAIL (2025-07-05 15:42:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.655923051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.625001583s FAIL (2025-07-05 15:41:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.625030117s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.677546247s FAIL (2025-07-05 15:40:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.677576945s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.668518247s FAIL (2025-07-05 15:38:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.668553393s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.604979387s FAIL (2025-07-05 15:38:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.60500215s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)