Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.600671041s FAIL (2025-08-28 14:12:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.600699105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.6802245s FAIL (2025-08-28 14:11:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.680257855s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.780427903s FAIL (2025-08-28 14:10:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.780453011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.864165567s FAIL (2025-08-28 14:10:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.864191115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.627788908s FAIL (2025-08-28 14:09:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.627814017s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.888414438s FAIL (2025-08-28 14:08:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.888441582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.414113375s FAIL (2025-08-28 14:07:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.414138926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.96326573s FAIL (2025-08-28 14:06:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.96329244s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.487841508s FAIL (2025-08-28 14:05:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.487871195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 7.046171146s FAIL (2025-08-28 14:04:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.046195993s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.66658103s FAIL (2025-08-28 14:03:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.666608231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.721627497s FAIL (2025-08-28 14:03:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.721682231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.555338137s FAIL (2025-08-28 14:02:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.55536588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.463271483s FAIL (2025-08-28 14:01:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.463299398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.183207295s FAIL (2025-08-28 14:00:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.183227684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.511909287s FAIL (2025-08-28 13:59:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.511937511s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.527104414s FAIL (2025-08-28 13:58:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.527129911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.370651864s FAIL (2025-08-28 13:58:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.370668307s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 8.300659216s FAIL (2025-08-28 13:57:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.300686358s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 10.651691468s FAIL (2025-08-28 13:56:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 10.651718551s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 7.093806629s FAIL (2025-08-28 13:55:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.093833661s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 13.057923725s FAIL (2025-08-28 13:54:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 13.057953201s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.827523681s FAIL (2025-08-28 13:53:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.827544772s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 8.715857852s FAIL (2025-08-28 13:52:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.715889102s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.979484634s FAIL (2025-08-28 13:51:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.979510554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 7.893346638s FAIL (2025-08-28 13:51:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.893373318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.689248602s FAIL (2025-08-28 13:50:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.689272508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 8.053603537s FAIL (2025-08-28 13:50:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.053625578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.83142538s FAIL (2025-08-28 13:49:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.831453743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 7.014448922s FAIL (2025-08-28 13:49:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.014476725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 7.105286243s FAIL (2025-08-28 13:48:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.105309086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.099424846s FAIL (2025-08-28 13:47:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.099448794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.085411219s FAIL (2025-08-28 13:47:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.085426667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.374240946s FAIL (2025-08-28 13:46:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.374260342s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.703852523s FAIL (2025-08-28 13:46:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.703884234s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.944190168s FAIL (2025-08-28 13:45:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.944223231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.573650603s FAIL (2025-08-28 13:44:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.573677795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.523287444s FAIL (2025-08-28 13:43:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.523317431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 8.783884488s FAIL (2025-08-28 13:42:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.783905697s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 12.462178666s FAIL (2025-08-28 13:41:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 12.4622019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.448450508s FAIL (2025-08-28 13:41:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.448479304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 8.032009943s FAIL (2025-08-28 13:40:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.032036954s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 7.047673446s FAIL (2025-08-28 13:40:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.047701039s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.569795475s FAIL (2025-08-28 13:39:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.569821383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.3109245s FAIL (2025-08-28 13:39:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 7.310952654s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.947780086s FAIL (2025-08-28 13:38:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 6.947812459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 8.301408315s FAIL (2025-08-28 13:37:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 8.301432021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00105401s FAIL (2025-08-28 13:36:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 28 13:36:30.497478686 (3h20m32.124513679s) [upload=3481]: Starting upload Aug 28 13:36:30.497550442 (3h20m32.124585435s) [upload=3481]: Done starting upload Aug 28 13:36:30.497677386 (3h20m32.124712379s) [upload=3481]: Uploading segments... Aug 28 13:36:30.497858151 (3h20m32.124893144s) [upload=3481]: Got next segment. Inline: false Aug 28 13:36:30.497920541 (3h20m32.124955524s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 13:36:30.497925360 (3h20m32.124960353s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 13:36:31.034622239 (3h20m32.661657222s) [seg_pos={0 0}, upload=3481]: Begin upload segment... Aug 28 13:36:31.034684969 (3h20m32.661719962s) [seg_pos={0 0}, upload=3481]: Done begin upload segment. Aug 28 13:36:31.034720607 (3h20m32.6617556s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.034740064 (3h20m32.661775057s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035108530 (3h20m32.662143523s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035388966 (3h20m32.662423959s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035443631 (3h20m32.662478624s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035777630 (3h20m32.662812623s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.035784693 (3h20m32.662819686s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036092412 (3h20m32.663127406s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.036769006 (3h20m32.663803999s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037007854 (3h20m32.664042847s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.037386979 (3h20m32.664421962s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.038302701 (3h20m32.665337694s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039587970 (3h20m32.666622963s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.039887253 (3h20m32.666922246s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040142282 (3h20m32.667177275s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:31.040381298 (3h20m32.667416322s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Uploading piece... Aug 28 13:36:32.792384165 (3h20m34.419419158s) [upload=3481]: Next returned nil segment Aug 28 13:36:32.792414774 (3h20m34.419449767s) [upload=3481]: Waiting for error group managing segments... Aug 28 13:36:33.092112022 (3h20m34.719147005s) [noise=true , address=104.36.82.13:4185 , piece_id=37LOVFG6H7N3FBFTZZJ7FMMHW6RVXIPNJ3XE4QHR4YANB6JUJ5AQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.156779538 (3h20m34.783814531s) [noise=true , address=104.36.82.6:4876 , piece_id=XYDTWDK2SD4GZA3WDOLD7ANDCS5K6W3P2QVRUETNHX252DN2ZDFQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.163104000 (3h20m34.790138993s) [noise=true , address=104.36.82.16:4294 , piece_id=ISAGFYCIRCYXWJYKZVOGYSWZSVBH2K67DSAHIC2WYCR44NZYGFAA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.245181378 (3h20m34.872216371s) [noise=true , address=104.36.82.4:4804 , piece_id=6Q4ZVEIBFIGQUFEWXLM3LVAYEZAONH5WX3XC2XO34LLOXLSKW5NQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.256380938 (3h20m34.883415931s) [noise=true , address=104.36.82.10:4077 , piece_id=WRWQUAQXWBMWV3VDFTBGP6NUKR4ZPUYZ2RKFQVDL6YYWPXK2HYDQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.282514012 (3h20m34.909549005s) [noise=true , address=45.91.116.8:4045 , piece_id=OGIBYXKMTVKVKSNKSXYAT4LSSFQ5PS2GKVPUP7IQKNRJETLSJU6A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.290816478 (3h20m34.917851481s) [noise=true , address=77.90.31.10:4517 , piece_id=7EG5RING2O562BZWDNUWFF2LC2YVNKWICVXSEJTKLU22RJ7TAGFA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.296202223 (3h20m34.923237216s) [noise=true , address=104.36.82.15:4258 , piece_id=OXEGYH3TTUCKDJMYPID2RT4DPAUM4PLINZB3XPMDNSZSRQD2Y4OQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.302510023 (3h20m34.929545016s) [noise=true , address=104.36.82.6:4875 , piece_id=POHGVYV2XS6IJ4JVX2ZWJZOK7WFCIMGL4VOZ5W4JMG6NZUEIZGKQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.314838241 (3h20m34.941873234s) [noise=true , address=104.36.82.9:4005 , piece_id=YF7QGTNDJAYBHMQUPLAIQCNIVK6SRROWJCJLZUZ4WEMB65QRSJDA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.324635829 (3h20m34.951670822s) [noise=true , address=104.36.82.2:4731 , piece_id=EJR7IKRAGYBEFUYJB6AVMYJAYALLUZ6XLTQTMME33T5YQ73DU76A, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.481717763 (3h20m35.108752756s) [noise=true , address=45.91.116.13:4155 , piece_id=JEJB3ZONNWHQNCQ5PNJSJLEGE3QANFKMTOPPPR56VZWYX27XKFXA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570703922 (3h20m35.197738915s) [noise=true , address=77.90.31.16:4734 , piece_id=BYLPWWVCOONEKXV2JKLJZR3RY7BN4CRCDSWHI7H7BDMSEVSLHGCQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: <nil> Aug 28 13:36:33.570728839 (3h20m35.197763832s) [seg_pos={0 0}, upload=3481]: Segment reached optimal threshold of 13 pieces. Aug 28 13:36:33.570892111 (3h20m35.197927104s) [noise=true , address=45.91.116.5:4297 , piece_id=3HWIQGBMO6UVX6XAK6UDQ6T6MSYOOAAUJ62EP4MWIBHJOHV5VFUQ, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.570948410 (3h20m35.197983403s) [noise=true , address=77.90.31.14:4662 , piece_id=HZ2FLTZMLHR2CPD3QAS4262CTIPFXOVDDMMAKIY5DHQL3ADNPKEA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571008464 (3h20m35.198043457s) [noise=true , address=45.91.116.14:4191 , piece_id=PCNFZWMUTUM6O2TE5DODOKWPULSFH2COPXJXRBVAPPV5DNG2S7IA, seg_pos={0 0}, upload=3481]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Aug 28 13:36:33.571034194 (3h20m35.198069187s) [seg_pos={0 0}, upload=3481]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 28 13:36:33.571118395 (3h20m35.198153388s) [upload=3481]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 13:36:33.571125267 (3h20m35.198160291s) [upload=3481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 13:36:33.970726284 (3h20m35.597761267s) [upload=3481]: Done uploading segments... Aug 28 13:36:33.970742806 (3h20m35.597777789s) [upload=3481]: Upload finished. err: <nil> Duration: 30.001066705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [...................X.............XX......XX...........X.....] duration: 2.125184046s FAIL (2025-08-28 10:11:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 10:08:11.274801933 (16h29m50.437402871s) [upload=17250]: Starting upload Aug 28 10:08:11.274863821 (16h29m50.437464759s) [upload=17250]: Done starting upload Aug 28 10:08:11.274957361 (16h29m50.437558299s) [upload=17250]: Uploading segments... Aug 28 10:08:11.275031934 (16h29m50.437632872s) [upload=17250]: Got next segment. Inline: true Aug 28 10:08:11.275036573 (16h29m50.437637511s) [upload=17250]: Waiting for error group managing segments... Aug 28 10:08:11.275076418 (16h29m50.437677356s) [upload=17250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 10:08:11.275080887 (16h29m50.437681825s) [upload=17250]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 10:08:11.275083592 (16h29m50.43768453s) [upload=17250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 10:08:11.561871034 (16h29m50.724471972s) [upload=17250]: Done uploading segments... Aug 28 10:08:11.561891493 (16h29m50.724492431s) [upload=17250]: Upload finished. err: <nil> Duration:* 2.125214905s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [....................X.............XX......XX...........X....] duration: 2.038227489s FAIL (2025-08-28 10:10:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 10:08:11.274801933 (16h29m50.437402871s) [upload=17250]: Starting upload Aug 28 10:08:11.274863821 (16h29m50.437464759s) [upload=17250]: Done starting upload Aug 28 10:08:11.274957361 (16h29m50.437558299s) [upload=17250]: Uploading segments... Aug 28 10:08:11.275031934 (16h29m50.437632872s) [upload=17250]: Got next segment. Inline: true Aug 28 10:08:11.275036573 (16h29m50.437637511s) [upload=17250]: Waiting for error group managing segments... Aug 28 10:08:11.275076418 (16h29m50.437677356s) [upload=17250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 10:08:11.275080887 (16h29m50.437681825s) [upload=17250]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 10:08:11.275083592 (16h29m50.43768453s) [upload=17250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 10:08:11.561871034 (16h29m50.724471972s) [upload=17250]: Done uploading segments... Aug 28 10:08:11.561891493 (16h29m50.724492431s) [upload=17250]: Upload finished. err: <nil> Duration:* 2.038256203s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)