Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X..X.......X..X........X....X..................X...........] duration: 5.664906351s FAIL (2025-08-09 12:57:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.664932621s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X....................X..........] duration: 1.933807228s FAIL (2025-08-09 12:57:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.933849479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..X..X.......X..X........X....X..................X..........] duration: 5.971996309s FAIL (2025-08-09 12:57:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.972023991s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X....................X.........] duration: 1.895689885s FAIL (2025-08-09 12:56:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.895721737s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X..X.......X..X........X....X..................X.........] duration: 5.725937331s FAIL (2025-08-09 12:56:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.725967749s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X.............................X....................X........] duration: 1.955406164s FAIL (2025-08-09 12:56:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.955436543s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X.............................X....................X.......] duration: 1.857045948s FAIL (2025-08-09 12:55:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.857071798s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X..X.......X..X........X....X..................X........] duration: 5.88283667s FAIL (2025-08-09 12:55:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.882918537s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X.............................X....................X......] duration: 1.840205698s FAIL (2025-08-09 12:55:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.840231828s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.778879984s FAIL (2025-08-09 12:55:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.778903148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X.............................X....................X.....] duration: 1.862845295s FAIL (2025-08-09 12:54:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.86287406s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.94781314s FAIL (2025-08-09 12:54:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.947841193s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....X..X.......X..X........X....X..................X.......] duration: 5.837078411s FAIL (2025-08-09 12:54:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.837104712s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X.............................X....................X....] duration: 1.845141502s FAIL (2025-08-09 12:54:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.845166419s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [......X..X.......X..X........X....X..................X......] duration: 5.644119565s FAIL (2025-08-09 12:53:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.644145925s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.877962876s FAIL (2025-08-09 12:53:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.877981721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X.............................X....................X...] duration: 1.882586748s FAIL (2025-08-09 12:53:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.882606295s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......X..X.......X..X........X....X..................X.....] duration: 6.017313575s FAIL (2025-08-09 12:53:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 6.017346058s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.842738881s FAIL (2025-08-09 12:52:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.842768888s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X.............................X....................X..] duration: 2.791352295s FAIL (2025-08-09 12:52:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 2.79140698s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [........X..X.......X..X........X....X..................X....] duration: 5.925235842s FAIL (2025-08-09 12:52:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.925262764s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.757289011s FAIL (2025-08-09 12:52:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.757312245s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X.............................X....................X.] duration: 1.881091205s FAIL (2025-08-09 12:51:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.881115891s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.725083193s FAIL (2025-08-09 12:51:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.725103382s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.........X..X.......X..X........X....X..................X...] duration: 5.779988393s FAIL (2025-08-09 12:51:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.780014142s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X.............................X....................X] duration: 1.604620866s FAIL (2025-08-09 12:50:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 10a1043c78271de769a080067e634c239cabf491e425dd62cce9f6ffc87592fd != before 09068e3519913bac65ae835ebbfb3837749812831166bbf3e75f86b1173f30ba ======LOGS====== Aug 9 12:50:45.382045598 (19h24m39.813125729s) [upload=20274]: Starting upload Aug 9 12:50:45.382098199 (19h24m39.81317833s) [upload=20274]: Done starting upload Aug 9 12:50:45.382149817 (19h24m39.813229958s) [upload=20274]: Uploading segments... Aug 9 12:50:45.382183722 (19h24m39.813263853s) [upload=20274]: Got next segment. Inline: true Aug 9 12:50:45.382187359 (19h24m39.81326749s) [upload=20274]: Waiting for error group managing segments... Aug 9 12:50:45.382224299 (19h24m39.81330443s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:50:45.382229058 (19h24m39.813309189s) [upload=20274]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:50:45.382231673 (19h24m39.813311804s) [upload=20274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:50:45.664009706 (19h24m40.095089827s) [upload=20274]: Done uploading segments... Aug 9 12:50:45.664029775 (19h24m40.095109906s) [upload=20274]: Upload finished. err: <nil> Duration:* 1.604647947s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 5.752454943s FAIL (2025-08-09 12:50:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.752490331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..........X..X.......X..X........X....X..................X..] duration: 5.926231557s FAIL (2025-08-09 12:50:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.926261053s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.717796452s FAIL (2025-08-09 12:50:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.717830768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [X..........X..X.......X..X........X....X..................X.] duration: 5.839580159s FAIL (2025-08-09 12:49:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 5.839607541s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.............................X....................] duration: 2.047243414s FAIL (2025-08-09 12:49:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.047276477s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X..........X..X.......X..X........X....X..................X] duration: 3.915074167s FAIL (2025-08-09 12:49:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 12:49:16.162475403 (19h23m10.593555534s) [upload=20251]: Starting upload Aug 9 12:49:16.162533975 (19h23m10.593614106s) [upload=20251]: Done starting upload Aug 9 12:49:16.162620190 (19h23m10.593700321s) [upload=20251]: Uploading segments... Aug 9 12:49:16.162706566 (19h23m10.593786697s) [upload=20251]: Got next segment. Inline: false Aug 9 12:49:16.162741021 (19h23m10.593821142s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:49:16.162745670 (19h23m10.593825801s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:49:16.665269286 (19h23m11.096349417s) [seg_pos={0 0}, upload=20251]: Begin upload segment... Aug 9 12:49:16.665385830 (19h23m11.096465951s) [seg_pos={0 0}, upload=20251]: Done begin upload segment. Aug 9 12:49:16.665411820 (19h23m11.096491951s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665444602 (19h23m11.096524723s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.665951772 (19h23m11.097031903s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666022388 (19h23m11.097102519s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666255574 (19h23m11.097335705s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666418966 (19h23m11.097499097s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.666835182 (19h23m11.097915303s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667102344 (19h23m11.098182475s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667459297 (19h23m11.098539428s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667731057 (19h23m11.098811188s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.667986004 (19h23m11.099066126s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668300447 (19h23m11.099380578s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668690843 (19h23m11.099770964s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.668997750 (19h23m11.100077882s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669247789 (19h23m11.10032792s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669486285 (19h23m11.100566416s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.669785598 (19h23m11.100865729s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:16.670163400 (19h23m11.101243531s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Uploading piece... Aug 9 12:49:17.774394222 (19h23m12.205474353s) [upload=20251]: Next returned nil segment Aug 9 12:49:17.774409772 (19h23m12.205489903s) [upload=20251]: Waiting for error group managing segments... Aug 9 12:49:18.072712493 (19h23m12.503792624s) [noise=true , address=64.49.13.169:4709 , piece_id=KGRIOHK2PPI5UOJ6OW2XT6QHEMO3J5ANMK4PHG3BHL42PAFFX6AQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.075331845 (19h23m12.506411976s) [noise=true , address=64.49.13.26:4955 , piece_id=IH24H6MPH45JE3MVJZFVKY5BGXJLPAOEUI5M55IEEJPOGZPRA4NQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.079297813 (19h23m12.510377944s) [noise=true , address=64.49.13.138:4487 , piece_id=QLIDB44FIHYPVMID3RPKO273M72CINRRGVGHSFHQBEZTOQGXSVWA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.619231717 (19h23m13.050311848s) [noise=true , address=104.36.82.47:4527 , piece_id=DJZOON4EEEEO2TCX77PTAFPJTW4IM7IMOEJYMUVYR4CQZQRUPVFA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.626572551 (19h23m13.057652682s) [noise=true , address=104.36.82.6:4902 , piece_id=T7SCA6IGDKOTLDOPFL23J4ERQIL6L2JSQAVF666ZWO6ZQWMZMSOA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.630244888 (19h23m13.061325019s) [noise=true , address=104.36.82.20:4457 , piece_id=NQPIQTBKBO3CXQZXZASDVC4WMEIO3N6C33OOICNIUI767SIL2RMA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.632311542 (19h23m13.063391673s) [noise=true , address=104.36.82.8:4990 , piece_id=2B7L2MJTBRIFRPIRHYDLX3LWHUGEAPL3SXSMPECCBA3DLVVKKA7Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.645595839 (19h23m13.07667596s) [noise=true , address=104.36.82.5:4849 , piece_id=U2CPGLQZHEP4WO52DNHM6TLDH5WSYMBGUV732DTYUZ3T6M4IZS6Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668704032 (19h23m13.099784163s) [noise=true , address=104.36.82.62:4428 , piece_id=QHLDBG4MU5UEOQKEP5BGLEIUDR4WIN2BVJ2G6JJCVCMMRK453TGQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.668906078 (19h23m13.099986209s) [noise=true , address=45.91.116.6:4005 , piece_id=UZMFSEKER3HKIYHRL3XLT3NYFWHM6FXWR3IJKZMS5LPRB4L4PUBA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.677864108 (19h23m13.108944229s) [noise=true , address=104.36.82.10:4082 , piece_id=D4JXITXH4CKUAZUK7U6K6NWB3BO6NQU7LTXMVY3CAIQXTQBSJXYA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.678095219 (19h23m13.10917535s) [noise=true , address=45.91.116.9:4265 , piece_id=TITGOVGLG7DGJRTTCACT5C74PQORWPLAZ23UYK7Z3QFBGE2CERVA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.679351924 (19h23m13.110432055s) [noise=true , address=104.36.82.11:4134 , piece_id=RNHD3WQOXUNV7UMRKVPPUYCLV26U2WLXPCM6PMOTAZBWVV2ML42Q, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680965711 (19h23m13.112045842s) [noise=true , address=104.36.82.22:4534 , piece_id=YIQYDHPSCM6TL7IKSVQ7WLNS6SZWMNZL3OKLNEJX3YWP5QJYBMPA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: <nil> Aug 9 12:49:18.680972605 (19h23m13.112052736s) [seg_pos={0 0}, upload=20251]: Segment reached optimal threshold of 14 pieces. Aug 9 12:49:18.681055504 (19h23m13.112135635s) [noise=true , address=104.36.82.23:4466 , piece_id=Y4I3CPRRCOJOHPJ2LYBJ54EL4IM43Z2DFPK3IMEBGIMUIUELSIAA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681123984 (19h23m13.112204115s) [noise=true , address=45.91.116.10:4087 , piece_id=W2KMCJVHCJY2ZJY6J4FP2UXGXC446MOUQAIIIH7NTM2C7OHCWJTA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125T5WzC4RDcp21oJ8zBMMBq5uw2X32Wf5FszZiWUdsSR95Pd2H): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681189911 (19h23m13.112270032s) [noise=true , address=104.36.82.38:4699 , piece_id=42L7TJ6TIKF2Z7E5AXZ2GVFZBVRCJWBSYCJKHU3VN5MOBK6KNYQA, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qMdRzPQnzcHfB45D28SRU2NMoaDDtXqkXgFjbvVRn7d8o2d5K): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681240949 (19h23m13.11232108s) [noise=true , address=104.36.82.46:4516 , piece_id=4VBHCGXKKRZE2BJCLMDJB5KJRLKAH2NTS5MJFMC33M7H4CTKLMHQ, seg_pos={0 0}, upload=20251]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FRYABJbf6iHTm3WiYcL192kFth6rHjrcMAS3ahF8ddm4ox6Uj): protocol: expected piece hash; context canceled; EOF Aug 9 12:49:18.681250296 (19h23m13.112330427s) [seg_pos={0 0}, upload=20251]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:49:18.681304049 (19h23m13.11238418s) [upload=20251]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:49:18.681307355 (19h23m13.112387486s) [upload=20251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:49:18.975455258 (19h23m13.406535379s) [upload=20251]: Done uploading segments... Aug 9 12:49:18.975470907 (19h23m13.406551038s) [upload=20251]: Upload finished. err: <nil> Duration: 3.915094667s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 5.711464556s FAIL (2025-08-09 12:49:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.711488893s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.............................X...................] duration: 1.837694175s FAIL (2025-08-09 12:48:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.837719122s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 5.818290936s FAIL (2025-08-09 12:48:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.818316374s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..X..........X..X.......X..X........X....X..................] duration: 5.775206962s FAIL (2025-08-09 12:48:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 12:35:26.630470350 (19h9m21.061550521s) [upload=20017]: Starting upload Aug 9 12:35:26.630561795 (19h9m21.061641956s) [upload=20017]: Done starting upload Aug 9 12:35:26.630676524 (19h9m21.061756655s) [upload=20017]: Uploading segments... Aug 9 12:35:26.631057663 (19h9m21.062137834s) [upload=20017]: Got next segment. Inline: false Aug 9 12:35:26.631119531 (19h9m21.062199693s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:35:26.631126856 (19h9m21.062207017s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:35:27.248644724 (19h9m21.679724855s) [seg_pos={0 0}, upload=20017]: Begin upload segment... Aug 9 12:35:27.248706221 (19h9m21.679786352s) [seg_pos={0 0}, upload=20017]: Done begin upload segment. Aug 9 12:35:27.248737561 (19h9m21.679817692s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.248773771 (19h9m21.679853902s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249205566 (19h9m21.680285698s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249517434 (19h9m21.680597565s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249844650 (19h9m21.680924771s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250169911 (19h9m21.681250043s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250361568 (19h9m21.681441689s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250488141 (19h9m21.681568272s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250656423 (19h9m21.681736554s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250747897 (19h9m21.681828028s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250942239 (19h9m21.68202237s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251024557 (19h9m21.682104688s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251230120 (19h9m21.682310251s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251285947 (19h9m21.682366068s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251575942 (19h9m21.682656074s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251842812 (19h9m21.682922943s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252066540 (19h9m21.683146671s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252295519 (19h9m21.68337565s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:28.391868714 (19h9m22.822948835s) [upload=20017]: Next returned nil segment Aug 9 12:35:28.391884463 (19h9m22.822964594s) [upload=20017]: Waiting for error group managing segments... Aug 9 12:35:28.690180201 (19h9m23.121260382s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:28.703816351 (19h9m23.134896482s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.174941963 (19h9m23.606022094s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.196691998 (19h9m23.627772139s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.200539399 (19h9m23.63161955s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.201593777 (19h9m23.632673898s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.206953251 (19h9m23.638033413s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.207582925 (19h9m23.638663056s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.209092193 (19h9m23.640172324s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.212293168 (19h9m23.643373309s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.231412579 (19h9m23.66249271s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.232120694 (19h9m23.663200815s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.247473538 (19h9m23.678553669s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276803756 (19h9m23.707883887s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276833984 (19h9m23.707914115s) [seg_pos={0 0}, upload=20017]: Segment reached optimal threshold of 14 pieces. Aug 9 12:35:29.276957630 (19h9m23.708037761s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277017624 (19h9m23.708097745s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Py6TMKBL8njtAK5Utj7kkXtVDQbvohnge7osJu8ewbVksodcu): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277120592 (19h9m23.708200723s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GMCu5o66NtZLav4dLpgbsM8SrD5SpL6GingUPMGphxKXrDHCc): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277180987 (19h9m23.708261118s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277201286 (19h9m23.708281417s) [seg_pos={0 0}, upload=20017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:35:29.277251162 (19h9m23.708331293s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:35:29.277254849 (19h9m23.70833498s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:35:29.590844830 (19h9m24.021924961s) [upload=20017]: Done uploading segments... Aug 9 12:35:29.590875468 (19h9m24.021955599s) [upload=20017]: Upload finished. err: <nil> Duration: 5.775233422s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.............................X..................] duration: 1.842236875s FAIL (2025-08-09 12:48:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.842263797s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.............................X.................] duration: 1.95128145s FAIL (2025-08-09 12:48:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.951350042s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.8493448s FAIL (2025-08-09 12:48:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.849373746s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X..........X..X.......X..X........X....X.................] duration: 5.846023607s FAIL (2025-08-09 12:47:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 12:35:26.630470350 (19h9m21.061550521s) [upload=20017]: Starting upload Aug 9 12:35:26.630561795 (19h9m21.061641956s) [upload=20017]: Done starting upload Aug 9 12:35:26.630676524 (19h9m21.061756655s) [upload=20017]: Uploading segments... Aug 9 12:35:26.631057663 (19h9m21.062137834s) [upload=20017]: Got next segment. Inline: false Aug 9 12:35:26.631119531 (19h9m21.062199693s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:35:26.631126856 (19h9m21.062207017s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:35:27.248644724 (19h9m21.679724855s) [seg_pos={0 0}, upload=20017]: Begin upload segment... Aug 9 12:35:27.248706221 (19h9m21.679786352s) [seg_pos={0 0}, upload=20017]: Done begin upload segment. Aug 9 12:35:27.248737561 (19h9m21.679817692s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.248773771 (19h9m21.679853902s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249205566 (19h9m21.680285698s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249517434 (19h9m21.680597565s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249844650 (19h9m21.680924771s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250169911 (19h9m21.681250043s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250361568 (19h9m21.681441689s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250488141 (19h9m21.681568272s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250656423 (19h9m21.681736554s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250747897 (19h9m21.681828028s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250942239 (19h9m21.68202237s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251024557 (19h9m21.682104688s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251230120 (19h9m21.682310251s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251285947 (19h9m21.682366068s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251575942 (19h9m21.682656074s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251842812 (19h9m21.682922943s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252066540 (19h9m21.683146671s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252295519 (19h9m21.68337565s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:28.391868714 (19h9m22.822948835s) [upload=20017]: Next returned nil segment Aug 9 12:35:28.391884463 (19h9m22.822964594s) [upload=20017]: Waiting for error group managing segments... Aug 9 12:35:28.690180201 (19h9m23.121260382s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:28.703816351 (19h9m23.134896482s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.174941963 (19h9m23.606022094s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.196691998 (19h9m23.627772139s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.200539399 (19h9m23.63161955s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.201593777 (19h9m23.632673898s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.206953251 (19h9m23.638033413s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.207582925 (19h9m23.638663056s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.209092193 (19h9m23.640172324s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.212293168 (19h9m23.643373309s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.231412579 (19h9m23.66249271s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.232120694 (19h9m23.663200815s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.247473538 (19h9m23.678553669s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276803756 (19h9m23.707883887s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276833984 (19h9m23.707914115s) [seg_pos={0 0}, upload=20017]: Segment reached optimal threshold of 14 pieces. Aug 9 12:35:29.276957630 (19h9m23.708037761s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277017624 (19h9m23.708097745s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Py6TMKBL8njtAK5Utj7kkXtVDQbvohnge7osJu8ewbVksodcu): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277120592 (19h9m23.708200723s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GMCu5o66NtZLav4dLpgbsM8SrD5SpL6GingUPMGphxKXrDHCc): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277180987 (19h9m23.708261118s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277201286 (19h9m23.708281417s) [seg_pos={0 0}, upload=20017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:35:29.277251162 (19h9m23.708331293s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:35:29.277254849 (19h9m23.70833498s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:35:29.590844830 (19h9m24.021924961s) [upload=20017]: Done uploading segments... Aug 9 12:35:29.590875468 (19h9m24.021955599s) [upload=20017]: Upload finished. err: <nil> Duration: 5.846051841s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X..........X..X.......X..X........X....X................] duration: 5.64934381s FAIL (2025-08-09 12:47:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 12:35:26.630470350 (19h9m21.061550521s) [upload=20017]: Starting upload Aug 9 12:35:26.630561795 (19h9m21.061641956s) [upload=20017]: Done starting upload Aug 9 12:35:26.630676524 (19h9m21.061756655s) [upload=20017]: Uploading segments... Aug 9 12:35:26.631057663 (19h9m21.062137834s) [upload=20017]: Got next segment. Inline: false Aug 9 12:35:26.631119531 (19h9m21.062199693s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:35:26.631126856 (19h9m21.062207017s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:35:27.248644724 (19h9m21.679724855s) [seg_pos={0 0}, upload=20017]: Begin upload segment... Aug 9 12:35:27.248706221 (19h9m21.679786352s) [seg_pos={0 0}, upload=20017]: Done begin upload segment. Aug 9 12:35:27.248737561 (19h9m21.679817692s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.248773771 (19h9m21.679853902s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249205566 (19h9m21.680285698s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249517434 (19h9m21.680597565s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249844650 (19h9m21.680924771s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250169911 (19h9m21.681250043s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250361568 (19h9m21.681441689s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250488141 (19h9m21.681568272s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250656423 (19h9m21.681736554s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250747897 (19h9m21.681828028s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250942239 (19h9m21.68202237s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251024557 (19h9m21.682104688s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251230120 (19h9m21.682310251s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251285947 (19h9m21.682366068s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251575942 (19h9m21.682656074s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251842812 (19h9m21.682922943s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252066540 (19h9m21.683146671s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252295519 (19h9m21.68337565s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:28.391868714 (19h9m22.822948835s) [upload=20017]: Next returned nil segment Aug 9 12:35:28.391884463 (19h9m22.822964594s) [upload=20017]: Waiting for error group managing segments... Aug 9 12:35:28.690180201 (19h9m23.121260382s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:28.703816351 (19h9m23.134896482s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.174941963 (19h9m23.606022094s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.196691998 (19h9m23.627772139s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.200539399 (19h9m23.63161955s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.201593777 (19h9m23.632673898s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.206953251 (19h9m23.638033413s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.207582925 (19h9m23.638663056s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.209092193 (19h9m23.640172324s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.212293168 (19h9m23.643373309s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.231412579 (19h9m23.66249271s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.232120694 (19h9m23.663200815s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.247473538 (19h9m23.678553669s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276803756 (19h9m23.707883887s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276833984 (19h9m23.707914115s) [seg_pos={0 0}, upload=20017]: Segment reached optimal threshold of 14 pieces. Aug 9 12:35:29.276957630 (19h9m23.708037761s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277017624 (19h9m23.708097745s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Py6TMKBL8njtAK5Utj7kkXtVDQbvohnge7osJu8ewbVksodcu): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277120592 (19h9m23.708200723s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GMCu5o66NtZLav4dLpgbsM8SrD5SpL6GingUPMGphxKXrDHCc): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277180987 (19h9m23.708261118s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277201286 (19h9m23.708281417s) [seg_pos={0 0}, upload=20017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:35:29.277251162 (19h9m23.708331293s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:35:29.277254849 (19h9m23.70833498s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:35:29.590844830 (19h9m24.021924961s) [upload=20017]: Done uploading segments... Aug 9 12:35:29.590875468 (19h9m24.021955599s) [upload=20017]: Upload finished. err: <nil> Duration: 5.649368307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.750442903s FAIL (2025-08-09 12:47:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.750470536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.............................X................] duration: 1.907037354s FAIL (2025-08-09 12:46:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.907062832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 5.73073801s FAIL (2025-08-09 12:46:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.730766514s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....X..........X..X.......X..X........X....X...............] duration: 5.874241954s FAIL (2025-08-09 12:46:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 12:35:26.630470350 (19h9m21.061550521s) [upload=20017]: Starting upload Aug 9 12:35:26.630561795 (19h9m21.061641956s) [upload=20017]: Done starting upload Aug 9 12:35:26.630676524 (19h9m21.061756655s) [upload=20017]: Uploading segments... Aug 9 12:35:26.631057663 (19h9m21.062137834s) [upload=20017]: Got next segment. Inline: false Aug 9 12:35:26.631119531 (19h9m21.062199693s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:35:26.631126856 (19h9m21.062207017s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:35:27.248644724 (19h9m21.679724855s) [seg_pos={0 0}, upload=20017]: Begin upload segment... Aug 9 12:35:27.248706221 (19h9m21.679786352s) [seg_pos={0 0}, upload=20017]: Done begin upload segment. Aug 9 12:35:27.248737561 (19h9m21.679817692s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.248773771 (19h9m21.679853902s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249205566 (19h9m21.680285698s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249517434 (19h9m21.680597565s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249844650 (19h9m21.680924771s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250169911 (19h9m21.681250043s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250361568 (19h9m21.681441689s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250488141 (19h9m21.681568272s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250656423 (19h9m21.681736554s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250747897 (19h9m21.681828028s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250942239 (19h9m21.68202237s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251024557 (19h9m21.682104688s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251230120 (19h9m21.682310251s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251285947 (19h9m21.682366068s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251575942 (19h9m21.682656074s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251842812 (19h9m21.682922943s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252066540 (19h9m21.683146671s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252295519 (19h9m21.68337565s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:28.391868714 (19h9m22.822948835s) [upload=20017]: Next returned nil segment Aug 9 12:35:28.391884463 (19h9m22.822964594s) [upload=20017]: Waiting for error group managing segments... Aug 9 12:35:28.690180201 (19h9m23.121260382s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:28.703816351 (19h9m23.134896482s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.174941963 (19h9m23.606022094s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.196691998 (19h9m23.627772139s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.200539399 (19h9m23.63161955s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.201593777 (19h9m23.632673898s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.206953251 (19h9m23.638033413s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.207582925 (19h9m23.638663056s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.209092193 (19h9m23.640172324s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.212293168 (19h9m23.643373309s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.231412579 (19h9m23.66249271s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.232120694 (19h9m23.663200815s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.247473538 (19h9m23.678553669s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276803756 (19h9m23.707883887s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276833984 (19h9m23.707914115s) [seg_pos={0 0}, upload=20017]: Segment reached optimal threshold of 14 pieces. Aug 9 12:35:29.276957630 (19h9m23.708037761s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277017624 (19h9m23.708097745s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Py6TMKBL8njtAK5Utj7kkXtVDQbvohnge7osJu8ewbVksodcu): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277120592 (19h9m23.708200723s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GMCu5o66NtZLav4dLpgbsM8SrD5SpL6GingUPMGphxKXrDHCc): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277180987 (19h9m23.708261118s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277201286 (19h9m23.708281417s) [seg_pos={0 0}, upload=20017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:35:29.277251162 (19h9m23.708331293s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:35:29.277254849 (19h9m23.70833498s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:35:29.590844830 (19h9m24.021924961s) [upload=20017]: Done uploading segments... Aug 9 12:35:29.590875468 (19h9m24.021955599s) [upload=20017]: Upload finished. err: <nil> Duration: 5.874267553s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.............................X...............] duration: 1.930749207s FAIL (2025-08-09 12:45:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.930775065s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [......X..........X..X.......X..X........X....X..............] duration: 5.859888164s FAIL (2025-08-09 12:45:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 12:35:26.630470350 (19h9m21.061550521s) [upload=20017]: Starting upload Aug 9 12:35:26.630561795 (19h9m21.061641956s) [upload=20017]: Done starting upload Aug 9 12:35:26.630676524 (19h9m21.061756655s) [upload=20017]: Uploading segments... Aug 9 12:35:26.631057663 (19h9m21.062137834s) [upload=20017]: Got next segment. Inline: false Aug 9 12:35:26.631119531 (19h9m21.062199693s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:35:26.631126856 (19h9m21.062207017s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 12:35:27.248644724 (19h9m21.679724855s) [seg_pos={0 0}, upload=20017]: Begin upload segment... Aug 9 12:35:27.248706221 (19h9m21.679786352s) [seg_pos={0 0}, upload=20017]: Done begin upload segment. Aug 9 12:35:27.248737561 (19h9m21.679817692s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.248773771 (19h9m21.679853902s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249205566 (19h9m21.680285698s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249517434 (19h9m21.680597565s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.249844650 (19h9m21.680924771s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250169911 (19h9m21.681250043s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250361568 (19h9m21.681441689s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250488141 (19h9m21.681568272s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250656423 (19h9m21.681736554s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250747897 (19h9m21.681828028s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.250942239 (19h9m21.68202237s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251024557 (19h9m21.682104688s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251230120 (19h9m21.682310251s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251285947 (19h9m21.682366068s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251575942 (19h9m21.682656074s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.251842812 (19h9m21.682922943s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252066540 (19h9m21.683146671s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:27.252295519 (19h9m21.68337565s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Uploading piece... Aug 9 12:35:28.391868714 (19h9m22.822948835s) [upload=20017]: Next returned nil segment Aug 9 12:35:28.391884463 (19h9m22.822964594s) [upload=20017]: Waiting for error group managing segments... Aug 9 12:35:28.690180201 (19h9m23.121260382s) [noise=true , address=64.49.13.137:4501 , piece_id=OCJFEDRZTUY2NYT36MBU5OIQ4KSTC74NHAFP3SVTOEY33JM6PGQQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:28.703816351 (19h9m23.134896482s) [noise=true , address=64.49.13.27:4968 , piece_id=V43KAMVBNJTO6R5UHH44LHUGVNP5GJDX4HHHV4XRALDM565LPUAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.174941963 (19h9m23.606022094s) [noise=true , address=45.91.116.11:4150 , piece_id=2ADPCNMUB5NIQGGPHM3OESTT567RBZ5ADIAS6YWJH5VJRITMGKQA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.196691998 (19h9m23.627772139s) [noise=true , address=77.90.31.16:4739 , piece_id=CTI5GSFL7JT766NU246USZIEKWNUHWNRMO5XKCP4YVJYJV5T4ACA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.200539399 (19h9m23.63161955s) [noise=true , address=104.36.82.36:4633 , piece_id=JA5HTSSWO464AVW3IG6VKWZMUJM4W4P42QU527QLF6PTXGIOAVPA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.201593777 (19h9m23.632673898s) [noise=true , address=104.36.82.46:4486 , piece_id=MVY534BAKHWA2TWF5255RFP3FN7MJ66HNM2R3H4LD3KEG635NMXA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.206953251 (19h9m23.638033413s) [noise=true , address=104.36.82.59:4317 , piece_id=5XKLIIH67AUKTZJGTJ6ACBLSJRGIDOYOEWLRK5ZHFKMQ5KSB2YAQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.207582925 (19h9m23.638663056s) [noise=true , address=104.36.82.9:4021 , piece_id=NUIJBRUMGNMU74CZPUYPNNZ5TBAUIS2DZI4L3GNKI3UMVGW7DZFQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.209092193 (19h9m23.640172324s) [noise=true , address=104.36.82.12:4174 , piece_id=JBZLZ6BDKM33Q7Z472WIZOFRBOWH3QO4SAQ4IAH5NBBV6EQKSZ4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.212293168 (19h9m23.643373309s) [noise=true , address=104.36.82.41:4780 , piece_id=37F7FJUHJ7GCP7UTKLNK2LBTBHAMP6BF7B6S5QLA4B2TPM3A7MUQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.231412579 (19h9m23.66249271s) [noise=true , address=104.36.82.43:4839 , piece_id=B7TMOALAE3KPGZVH6IDPSZ3SYDKIDMZM4ZJZHPFF4HVTWUWVP2PA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.232120694 (19h9m23.663200815s) [noise=true , address=104.36.82.34:4546 , piece_id=D6TN3JMCFGBB22IX5COHLTPMIXJKJWSAFDMVH7UQPLFKXBRWGL4A, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.247473538 (19h9m23.678553669s) [noise=true , address=104.36.82.20:4469 , piece_id=TU27MJAS4JECG33JNVZ274NCY6FBG3QC5GG2SNI23NOC5YEZKHGA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276803756 (19h9m23.707883887s) [noise=true , address=104.36.82.23:4451 , piece_id=4FB7J4KJVHDFZVZD7DTQN5JPG4BB4O473NWLUU6VKIZBYWJT5KOA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: <nil> Aug 9 12:35:29.276833984 (19h9m23.707914115s) [seg_pos={0 0}, upload=20017]: Segment reached optimal threshold of 14 pieces. Aug 9 12:35:29.276957630 (19h9m23.708037761s) [noise=true , address=104.36.82.35:4610 , piece_id=R276YFHNZUB2A2WQYYRBPYFZNBNBWUN4FC44HYW6WIAULGZX34YQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277017624 (19h9m23.708097745s) [noise=true , address=104.36.82.61:4404 , piece_id=ZPRR7V6R5PK4OQ6KLNNJ5LB4BGY75XTNRDVY2UFWP63R5CC6QPOQ, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Py6TMKBL8njtAK5Utj7kkXtVDQbvohnge7osJu8ewbVksodcu): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277120592 (19h9m23.708200723s) [noise=true , address=104.36.82.8:4984 , piece_id=J4MQQQLHFYG4RDRATUG4BDMWJ5CPZOLMZTOMPDWF5I53QQP5O26Q, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GMCu5o66NtZLav4dLpgbsM8SrD5SpL6GingUPMGphxKXrDHCc): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277180987 (19h9m23.708261118s) [noise=true , address=104.36.82.58:4276 , piece_id=A5LWMICRC7TQMY3Q2XIJRVE4UTXJEGHIUSOHZURTSJAD2UR7L7MA, seg_pos={0 0}, upload=20017]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126tQjCVpoLF4zbVbNH37ZTVbRRt2EK3zndnitxZs5xeQj2CzWr): protocol: expected piece hash; context canceled; EOF Aug 9 12:35:29.277201286 (19h9m23.708281417s) [seg_pos={0 0}, upload=20017]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 12:35:29.277251162 (19h9m23.708331293s) [upload=20017]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 12:35:29.277254849 (19h9m23.70833498s) [upload=20017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:35:29.590844830 (19h9m24.021924961s) [upload=20017]: Done uploading segments... Aug 9 12:35:29.590875468 (19h9m24.021955599s) [upload=20017]: Upload finished. err: <nil> Duration: 5.859910136s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 6.000955458s FAIL (2025-08-09 12:45:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.000981207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 5.766424339s FAIL (2025-08-09 12:45:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049416411 (18h46m6.480496542s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049427101 (18h46m6.480507232s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049751782 (18h46m6.480831903s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.049793271 (18h46m6.480873392s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050172537 (18h46m6.481252668s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.050791842 (18h46m6.481871963s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051017504 (18h46m6.482097635s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051185014 (18h46m6.482265145s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051309121 (18h46m6.482389252s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051635085 (18h46m6.482715206s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:12.051769001 (18h46m6.482849122s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Uploading piece... Aug 9 12:12:13.785898550 (18h46m8.216978681s) [upload=19606]: Next returned nil segment Aug 9 12:12:13.785913058 (18h46m8.216993189s) [upload=19606]: Waiting for error group managing segments... Aug 9 12:12:14.140433860 (18h46m8.571513991s) [noise=true , address=104.36.82.9:4006 , piece_id=J64YVKOG73B355KPNNXRFWKWSVNKMKPSRCYEEQIZTJ5VPPZHIZRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.158773539 (18h46m8.58985368s) [noise=true , address=104.36.82.7:4912 , piece_id=6GFZ5IJR7C4CYXN7TTMZ54NMIKDXSE6U5Q6XYSZTXVUCXDLEJWMA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.170688336 (18h46m8.601768457s) [noise=true , address=104.36.82.15:4257 , piece_id=MLCVRW7UQXH5YWAE5W56NBHPRG3HOTF22KRY4CDT2XGWSZV7TP5A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.203613310 (18h46m8.634693441s) [noise=true , address=104.36.82.15:4258 , piece_id=L2G72WFB5YXUZLSSRYHQ3BPJJ4CTSHYA65DH5VS4DHVA2XAQZSEA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.212362059 (18h46m8.64344219s) [noise=true , address=104.36.82.10:4079 , piece_id=VFU44CPWCYKKA5UZPAH2AH5Y5YF3W2PNEJOFX5Y7AIA6XIVIPUTQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.228742190 (18h46m8.659822341s) [noise=true , address=104.36.82.12:4151 , piece_id=HE4JIP73GL67MA5G7QBLAQKHTDJQ6ATW7PTWQTKQZNJ4GAD2LZ3A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:14.233030845 (18h46m8.664110976s) [noise=true , address=104.36.82.9:4007 , piece_id=MQ5JIWZ7XUISPF6GAPJYJCO43OQPXON7RPWKXSRFXB2U2CT7B2YA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: <nil> Aug 9 12:12:41.164497867 (18h46m35.595577998s) [noise=true , address=45.91.116.5:4297 , piece_id=LSU2IQLCISQDM3WPYDJPIGTGXUFE3ABKUILBBUV3X77H4ZZWYWTA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59218->45.91.116.5:4297: use of closed network connection Aug 9 12:12:41.164599561 (18h46m35.595679692s) [noise=true , address=45.91.116.6:4973 , piece_id=A6KDKLI2RD4VGJPIACPAZQSKC5XNK5JOT2SBUGJA4OM3WGDRP74A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42486->45.91.116.6:4973: use of closed network connection Aug 9 12:12:41.164656500 (18h46m35.595736641s) [noise=true , address=77.90.31.10:4518 , piece_id=CH43PJSZ5WESX4YCEEHDSN4OZXEHC7PHR3DHYEGJ3FISUB6Y7FLQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39074->77.90.31.10:4518: use of closed network connection Aug 9 12:12:41.164678142 (18h46m35.595758273s) [noise=true , address=45.91.116.8:4047 , piece_id=ZZF43C6KSYWMZ5YH5JBX7UAPHRPCM57YASB4UNVXBSVDHZQG5VZA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50522->45.91.116.8:4047: use of closed network connection Aug 9 12:12:41.164730031 (18h46m35.595810172s) [noise=true , address=45.91.116.13:4155 , piece_id=OZSGSJSOLPSPABEGXMNBC2RJDKKVFQNGTSI7VSVJOR4SJK4LMSUQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51938->45.91.116.13:4155: use of closed network connection Aug 9 12:12:41.164778003 (18h46m35.595858134s) [noise=true , address=77.90.31.17:4771 , piece_id=4HBKWZZQHDKSZ6LTQSVP5NAJ4425655SHA4VDAYXIIZKZSE334XA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44716->77.90.31.17:4771: use of closed network connection Aug 9 12:12:41.164806728 (18h46m35.595886869s) [noise=true , address=77.90.31.10:4517 , piece_id=XM2FGSE6TOPHTO7BXUYL6UHSFPCO7GOSNR4HPB4TXUJSBYY64SPA, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42908->77.90.31.10:4517: use of closed network connection Aug 9 12:12:41.164819602 (18h46m35.595899824s) [noise=true , address=45.91.116.4:4937 , piece_id=72QZTOXBOTYN2GP2BAPV2UIRX457Y2X3R34QDNREMTC7WQWQHPRQ, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55514->45.91.116.4:4937: use of closed network connection Aug 9 12:12:41.164867935 (18h46m35.595948066s) [noise=true , address=77.90.31.16:4735 , piece_id=IWCRGDTBTEMBVXV2WVHBK2WLN3LPOYQHZVLYOZKWCMIGLRWYUR6A, seg_pos={0 0}, upload=19606]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51694->77.90.31.16:4735: use of closed network connection Aug 9 12:12:41.164894456 (18h46m35.595974587s) [seg_pos={0 0}, upload=19606]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 9 12:12:41.398470245 (18h46m35.829550376s) [upload=19606]: Done uploading segments... Aug 9 12:12:41.398495032 (18h46m35.829575163s) [upload=19606]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.766449698s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.............................X..............] duration: 1.870013349s FAIL (2025-08-09 12:44:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 12:33:43.830000913 (19h7m38.261081044s) [upload=19989]: Starting upload Aug 9 12:33:43.830045678 (19h7m38.261125799s) [upload=19989]: Done starting upload Aug 9 12:33:43.830092328 (19h7m38.261172459s) [upload=19989]: Uploading segments... Aug 9 12:33:43.830128096 (19h7m38.261208217s) [upload=19989]: Got next segment. Inline: true Aug 9 12:33:43.830131733 (19h7m38.261211864s) [upload=19989]: Waiting for error group managing segments... Aug 9 12:33:43.830165819 (19h7m38.26124595s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 12:33:43.830169886 (19h7m38.261250018s) [upload=19989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 12:33:43.830172822 (19h7m38.261252953s) [upload=19989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 12:33:44.042197668 (19h7m38.473277799s) [upload=19989]: Done uploading segments... Aug 9 12:33:44.042213448 (19h7m38.473293579s) [upload=19989]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.870042195s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)