Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.928309338s FAIL (2025-07-02 06:23:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.92833123s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.67037878s FAIL (2025-07-02 06:23:23 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.670410651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.644895195s FAIL (2025-07-02 06:22:51 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.6449292s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.743815797s FAIL (2025-07-02 06:22:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.743840334s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.08592503s FAIL (2025-07-02 06:22:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.085952662s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.637867935s FAIL (2025-07-02 06:22:17 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.637903212s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.035821532s FAIL (2025-07-02 06:21:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.035845107s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.71738766s FAIL (2025-07-02 06:21:03 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.717411255s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.763326377s FAIL (2025-07-02 06:20:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.763350294s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.72360173s FAIL (2025-07-02 06:20:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.723626176s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.681550658s FAIL (2025-07-02 06:19:55 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.681576046s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.877582739s FAIL (2025-07-02 06:19:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.877602167s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.598106498s FAIL (2025-07-02 06:18:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.598133088s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.87635831s FAIL (2025-07-02 06:18:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.876382436s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.656223612s FAIL (2025-07-02 06:18:19 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.656256314s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.228414602s FAIL (2025-07-02 06:17:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.228444449s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.602302655s FAIL (2025-07-02 06:17:19 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.602328155s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.871125266s FAIL (2025-07-02 06:16:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.871149542s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.697769106s FAIL (2025-07-02 06:16:36 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.697801378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.629829675s FAIL (2025-07-02 06:16:17 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.629856185s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.229339169s FAIL (2025-07-02 06:15:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.229353506s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.667208927s FAIL (2025-07-02 06:15:44 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.667230158s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.217020052s FAIL (2025-07-02 06:15:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.217042584s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.679599914s FAIL (2025-07-02 06:15:11 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.679623408s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.148431608s FAIL (2025-07-02 06:14:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.148487183s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.631883066s FAIL (2025-07-02 06:14:47 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.631902051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.628224783s FAIL (2025-07-02 06:14:16 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.628249301s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.333689583s FAIL (2025-07-02 06:13:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.33372988s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.782990324s FAIL (2025-07-02 06:13:21 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.78301915s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.83583223s FAIL (2025-07-02 06:13:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.835857619s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.630033504s FAIL (2025-07-02 06:12:53 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.630059785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.620617777s FAIL (2025-07-02 06:12:37 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.620642776s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.228615461s FAIL (2025-07-02 06:12:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.228671237s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.662723016s FAIL (2025-07-02 06:12:17 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.662746612s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.379283899s FAIL (2025-07-02 06:11:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.379324245s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.926006254s FAIL (2025-07-02 06:11:30 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.926034529s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.117726872s FAIL (2025-07-02 06:10:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.11774158s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.699273926s FAIL (2025-07-02 06:10:25 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.699298364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.826341552s FAIL (2025-07-02 06:10:05 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.826363965s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.529430296s FAIL (2025-07-02 06:09:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.529451326s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.656084483s FAIL (2025-07-02 06:09:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.656115914s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.621362343s FAIL (2025-07-02 06:09:06 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.621393102s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.992770045s FAIL (2025-07-02 06:08:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.992787699s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.610260736s FAIL (2025-07-02 06:08:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.610292205s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.814458985s FAIL (2025-07-02 06:08:11 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.814506676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.770667728s FAIL (2025-07-02 06:07:30 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.770702334s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.906088492s FAIL (2025-07-02 06:07:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.906115483s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.670350525s FAIL (2025-07-02 06:07:09 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 33057701s) [noise=true , address=77.90.31.21:4877 , piece_id=MIQJKZPIULHKQYFT5ANV3ULTEULC6V6FBGT4GBBFKE75SJHA5P5A, seg_pos={0 0}, upload=170077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 2 05:49:27.162458798 (212h39m1.933100173s) [seg_pos={0 0}, upload=170077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:49:27.162604125 (212h39m1.933245511s) [upload=170077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:49:27.162612772 (212h39m1.933254147s) [upload=170077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:49:27.459834790 (212h39m2.230476125s) [upload=170077]: Done uploading segments... Jul 2 05:49:27.459858125 (212h39m2.23049946s) [upload=170077]: Upload finished. err: <nil> Jul 2 05:50:10.576443205 (212h39m45.34708454s) [upload=170087]: Starting upload Jul 2 05:50:10.576578824 (212h39m45.347220159s) [upload=170087]: Done starting upload Jul 2 05:50:10.576656032 (212h39m45.347297367s) [upload=170087]: Uploading segments... Jul 2 05:50:10.576828953 (212h39m45.347470288s) [upload=170087]: Got next segment. Inline: false Jul 2 05:50:10.576859220 (212h39m45.347500556s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 05:50:10.576864240 (212h39m45.347505575s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 05:50:10.808702028 (212h39m45.579343363s) [seg_pos={0 0}, upload=170087]: Begin upload segment... Jul 2 05:50:10.808752013 (212h39m45.579393348s) [seg_pos={0 0}, upload=170087]: Done begin upload segment. Jul 2 05:50:10.808775979 (212h39m45.579417314s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.808808541 (212h39m45.579449887s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809108976 (212h39m45.579750311s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809199409 (212h39m45.579840744s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809343404 (212h39m45.579984739s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809468484 (212h39m45.580109819s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809566601 (212h39m45.580207936s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809741396 (212h39m45.580382731s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809786091 (212h39m45.580427426s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.809987076 (212h39m45.580628411s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810154466 (212h39m45.580795801s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810254087 (212h39m45.580895422s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810444401 (212h39m45.581085736s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810549342 (212h39m45.581190677s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810639985 (212h39m45.58128132s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:10.810836050 (212h39m45.581477385s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Uploading piece... Jul 2 05:50:12.497702968 (212h39m47.268344303s) [upload=170087]: Next returned nil segment Jul 2 05:50:12.497758344 (212h39m47.268399679s) [upload=170087]: Waiting for error group managing segments... Jul 2 05:50:12.786708957 (212h39m47.557350282s) [noise=true , address=104.36.82.9:4005 , piece_id=VCDXR7AP5LAQYG3DGOQXG4VE2DBZPTFUFTC5UG62O2Q2EUOLU3JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.786763041 (212h39m47.557404376s) [noise=true , address=104.36.82.2:4730 , piece_id=K76VXRB53GNZY64MN6VV542BEUEDKETCBHZKVFR75B672LYLMY6Q, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.806312102 (212h39m47.576953437s) [noise=true , address=104.36.82.6:4873 , piece_id=UA5XWYUASVTQO3DYTTMXDX7XNI567AC5GDPOMCP7VZ3NBWTORHCA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.824384578 (212h39m47.595025913s) [noise=true , address=104.36.82.14:4221 , piece_id=OTNAXF723VCNEP3Y3FISABPXDXIYZW7WETIOI6ET4ISTFRXDH42A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.838516715 (212h39m47.60915805s) [noise=true , address=104.36.82.16:4294 , piece_id=DSC3LD6TEAFJG4PZGOE5XJRHL6DGOB23BJKYG6OXUF4FHV57ITOQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842143875 (212h39m47.61278521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZHVI5L2VO2AIOMK4A5V32AXQ5OXDOMZVRAL4J5FHNWRPUGAURM5A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.842170876 (212h39m47.612812211s) [noise=true , address=104.36.82.14:4223 , piece_id=T47ENG4WLEYUUUC5UHUE5H7DGZFOHN75MCZKGBRCKHVF62H74R2A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.847660860 (212h39m47.618302195s) [noise=true , address=45.91.116.7:4009 , piece_id=TZ35AEP3ULZIUZENBLPS7OH6UPXVGAU2ALCJNPURLSLPDKYXGWVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.849814209 (212h39m47.620455544s) [noise=true , address=45.91.116.8:4047 , piece_id=A5H7DDPS3F5HAQ3IGU64GCAZ4SGQPBHUMKHUIO2KDKMDEMYED6KQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.868470061 (212h39m47.639111396s) [noise=true , address=77.90.31.11:4555 , piece_id=A5C7MNN2SCQTAOAX7DAZ75IU2NUHIOEIRVMMA3YBT66ADX4L245A, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912562127 (212h39m47.683203472s) [noise=true , address=45.91.116.6:4975 , piece_id=RLDSR5V7NKOPW3CFVECKWFVANLWRILHOGKLEZJZM4HA622ZL6CBQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.912882240 (212h39m47.683523575s) [noise=true , address=45.91.116.12:4225 , piece_id=O3CEYCZBQNUN3JBRI4XQNVEW6QFPWVWPARMSNMHQXT5XOPTIKKIQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.917969412 (212h39m47.688610757s) [noise=true , address=45.91.116.13:4154 , piece_id=EJZ3TEZTKTIWIVQBMPZ2DXVBK2Z3SYVC4OJ2CAOUZ2YYNKCA5RWA, seg_pos={0 0}, upload=170087]: Done uploading piece. err: <nil> Jul 2 05:50:12.918003127 (212h39m47.688644482s) [seg_pos={0 0}, upload=170087]: Segment reached optimal threshold of 13 pieces. Jul 2 05:50:12.918257053 (212h39m47.688898388s) [noise=true , address=45.91.116.5:4299 , piece_id=BGBI64TH63M5CVR2NW4KRIYBKNGACEVHLCKXWMWWHJ5JJPYSSRKQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918269927 (212h39m47.688911262s) [noise=true , address=45.91.116.10:4081 , piece_id=O2TURAXXQM4TXMTEKE557QI5EZORTXAH3VEPSCSTDUXNL6RCMQVQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918347345 (212h39m47.68898868s) [noise=true , address=77.90.31.17:4770 , piece_id=2TT5RR5GBUT7TK2PRNCOHFCVVPDELCDNXH4RHIQM5UD2EJ5VY2JQ, seg_pos={0 0}, upload=170087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Jul 2 05:50:12.918369819 (212h39m47.689011154s) [seg_pos={0 0}, upload=170087]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 05:50:12.918447417 (212h39m47.689088752s) [upload=170087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 05:50:12.918451425 (212h39m47.68909275s) [upload=170087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 05:52:29.749365718 (212h42m4.520007053s) [upload=170087]: Done uploading segments... Jul 2 05:52:29.749385846 (212h42m4.520027182s) [upload=170087]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.670376907s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.84720671s FAIL (2025-07-02 06:06:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.847220476s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000760002s FAIL (2025-07-02 06:06:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000783797s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)