Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 423724 reports (Page 8201 of 8475)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [............................XXX.X.X.........................]
duration: 
5.824467112s
FAIL (2025-07-09 00:58:14 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.824485767s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [....X...............X...X..X.X.X..X.X...XX....X.....X...X...]
duration: 
20.585709385s
FAIL (2025-07-09 00:58:01 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.58572778s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
20 out of 60 is failed (threshold=3) [XXXX.X....X..XXXXXXX.XXXX.XXX...............................]
duration: 
41.25486366s
FAIL (2025-07-09 00:57:59 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 41.254883698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.............................XXX.X.X........................]
duration: 
6.016582856s
FAIL (2025-07-09 00:57:04 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.016608704s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.....X...............X...X..X.X.X..X.X...XX....X.....X...X..]
duration: 
6.311860966s
FAIL (2025-07-09 00:56:59 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.311882446s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [XXXXX.X....X..XXXXXXX.XXXX.XXX..............................]
duration: 
29.766466254s
FAIL (2025-07-09 00:56:56 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.766496473s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..............................XXX.X.X.......................]
duration: 
5.764108172s
FAIL (2025-07-09 00:56:18 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.764135605s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
22 out of 60 is failed (threshold=3) [XXXXXX.X....X..XXXXXXX.XXXX.XXX.............................]
duration: 
40.631319849s
FAIL (2025-07-09 00:56:04 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 40.631331241s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [......X...............X...X..X.X.X..X.X...XX....X.....X...X.]
duration: 
6.287062216s
FAIL (2025-07-09 00:55:51 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.28709597s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [...............................XXX.X.X......................]
duration: 
17.845464038s
FAIL (2025-07-09 00:55:24 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 17.845488654s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [X......X...............X...X..X.X.X..X.X...XX....X.....X...X]
duration: 
30.000789082s
FAIL (2025-07-09 00:55:23 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000807056s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
23 out of 60 is failed (threshold=3) [XXXXXXX.X....X..XXXXXXX.XXXX.XXX............................]
duration: 
39.180001259s
FAIL (2025-07-09 00:55:23 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 39.180021959s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [................................XXX.X.X.....................]
duration: 
5.868151125s
FAIL (2025-07-09 00:54:41 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.868167788s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.................................XXX.X.X....................]
duration: 
26.638714394s
FAIL (2025-07-09 00:54:35 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 26.638741607s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
24 out of 60 is failed (threshold=3) [XXXXXXXX.X....X..XXXXXXX.XXXX.XXX...........................]
duration: 
24.052486306s
FAIL (2025-07-09 00:54:29 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 24.052498879s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
25 out of 60 is failed (threshold=3) [XXXXXXXXX.X....X..XXXXXXX.XXXX.XXX..........................]
duration: 
37.560810892s
FAIL (2025-07-09 00:54:04 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 37.560823566s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.X......X...............X...X..X.X.X..X.X...XX....X.....X...]
duration: 
6.389070548s
FAIL (2025-07-09 00:53:47 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.389095627s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
26 out of 60 is failed (threshold=3) [XXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX.........................]
duration: 
39.504245282s
FAIL (2025-07-09 00:53:27 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 39.504324013s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..................................XXX.X.X...................]
duration: 
5.798982948s
FAIL (2025-07-09 00:53:23 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.799007925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [...................................XXX.X.X..................]
duration: 
21.912550128s
FAIL (2025-07-09 00:53:13 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 21.912581057s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [..X......X...............X...X..X.X.X..X.X...XX....X.....X..]
duration: 
21.878022703s
FAIL (2025-07-09 00:53:04 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.878042811s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
27 out of 60 is failed (threshold=3) [XXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX........................]
duration: 
42.669989711s
FAIL (2025-07-09 00:52:47 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 42.670063603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [....................................XXX.X.X.................]
duration: 
5.832278052s
FAIL (2025-07-09 00:52:33 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.832310413s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
28 out of 60 is failed (threshold=3) [XXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX.......................]
duration: 
29.681947077s
FAIL (2025-07-09 00:52:04 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.68196436s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.....................................XXX.X.X................]
duration: 
6.175345113s
FAIL (2025-07-09 00:52:03 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.175367957s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [...X......X...............X...X..X.X.X..X.X...XX....X.....X.]
duration: 
7.340126667s
FAIL (2025-07-09 00:51:45 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.34014295s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [....X......X...............X...X..X.X.X..X.X...XX....X.....X]
duration: 
30.001466065s
FAIL (2025-07-09 00:51:37 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001491021s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX......................]
duration: 
34.432726686s
FAIL (2025-07-09 00:51:34 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 34.432749148s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [......................................XXX.X.X...............]
duration: 
6.13215709s
FAIL (2025-07-09 00:50:50 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.13218382s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX.....................]
duration: 
43.862554599s
FAIL (2025-07-09 00:50:47 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 43.862566051s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [X....X......X...............X...X..X.X.X..X.X...XX....X.....]
duration: 
21.050370211s
FAIL (2025-07-09 00:50:20 UTC) 
orbiter warning: context deadline exceeded
Duration: 21.050389918s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.......................................XXX.X.X..............]
duration: 
5.754783283s
FAIL (2025-07-09 00:50:14 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.75481296s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX....................]
duration: 
30.346071869s
FAIL (2025-07-09 00:50:03 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.346083391s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.X....X......X...............X...X..X.X.X..X.X...XX....X....]
duration: 
6.371877136s
FAIL (2025-07-09 00:49:37 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.371901092s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX...................]
duration: 
45.279288271s
FAIL (2025-07-09 00:49:32 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 45.279305174s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [........................................XXX.X.X.............]
duration: 
5.807819181s
FAIL (2025-07-09 00:49:29 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.80783817s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [..X....X......X...............X...X..X.X.X..X.X...XX....X...]
duration: 
6.072784031s
FAIL (2025-07-09 00:49:09 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.072807275s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.........................................XXX.X.X............]
duration: 
5.831033117s
FAIL (2025-07-09 00:49:07 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.83107089s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX..................]
duration: 
39.29241898s
FAIL (2025-07-09 00:48:47 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 39.292439008s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..........................................XXX.X.X...........]
duration: 
7.168262259s
FAIL (2025-07-09 00:48:25 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.168281115s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [X..X....X......X...............X...X..X.X.X..X.X...XX....X..]
duration: 
5.937531795s
FAIL (2025-07-09 00:48:11 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.937552034s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [...........................................XXX.X.X..........]
duration: 
5.899639601s
FAIL (2025-07-09 00:48:06 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.899656963s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [.X..X....X......X...............X...X..X.X.X..X.X...XX....X.]
duration: 
28.426954871s
FAIL (2025-07-09 00:48:00 UTC) 
orbiter warning: context deadline exceeded
Duration: 28.42697571s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [............................................XXX.X.X.........]
duration: 
18.953597s
FAIL (2025-07-09 00:47:58 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 18.9536235s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX.................]
duration: 
49.447672847s
FAIL (2025-07-09 00:47:49 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 49.447686333s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.............................................XXX.X.X........]
duration: 
14.511820619s
FAIL (2025-07-09 00:47:26 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 14.511845737s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
15 out of 60 is failed (threshold=3) [X.X..X....X......X...............X...X..X.X.X..X.X...XX....X]
duration: 
30.000693207s
FAIL (2025-07-09 00:47:26 UTC) 
orbiter warning: context deadline exceeded
Duration: 30.00073098s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
35 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX................]
duration: 
39.447894656s
FAIL (2025-07-09 00:46:59 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 39.447909976s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..............................................XXX.X.X.......]
duration: 
25.507005673s
FAIL (2025-07-09 00:46:26 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
39:26.363507162 (375h29m1.134148507s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:39:26.363513754 (375h29m1.134155089s) [upload=300391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:39:26.664388326 (375h29m1.435029661s) [upload=300391]: Done uploading segments...
Jul  9 00:39:26.664407853 (375h29m1.435049188s) [upload=300391]: Upload finished. err: <nil>
Jul  9 00:40:22.831115797 (375h29m57.601757142s) [upload=300409]: Starting upload
Jul  9 00:40:22.831211139 (375h29m57.601852484s) [upload=300409]: Done starting upload
Jul  9 00:40:22.831297705 (375h29m57.60193906s) [upload=300409]: Uploading segments...
Jul  9 00:40:22.831515621 (375h29m57.602156976s) [upload=300409]: Got next segment. Inline: false
Jul  9 00:40:22.831567641 (375h29m57.602208996s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:40:22.831574394 (375h29m57.60221573s) [upload=300409]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:40:23.136866206 (375h29m57.907507541s) [seg_pos={0 0}, upload=300409]: Begin upload segment...
Jul  9 00:40:23.136907616 (375h29m57.907548951s) [seg_pos={0 0}, upload=300409]: Done begin upload segment.
Jul  9 00:40:23.136933185 (375h29m57.90757452s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.136949325 (375h29m57.90759066s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137300748 (375h29m57.907942083s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137357226 (375h29m57.907998551s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137573079 (375h29m57.908214414s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137669523 (375h29m57.908310858s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137935562 (375h29m57.908576898s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.137991359 (375h29m57.908632694s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138195048 (375h29m57.908836384s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138296653 (375h29m57.908937988s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138457411 (375h29m57.909098746s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138578562 (375h29m57.909219887s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138658365 (375h29m57.9092997s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138822549 (375h29m57.909463884s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.138937289 (375h29m57.909578614s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:23.139137432 (375h29m57.909778767s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Uploading piece...
Jul  9 00:40:24.925431157 (375h29m59.696072493s) [upload=300409]: Next returned nil segment
Jul  9 00:40:24.925452939 (375h29m59.696094284s) [upload=300409]: Waiting for error group managing segments...
Jul  9 00:40:25.211762407 (375h29m59.982403742s) [noise=true , address=45.91.116.11:4118    , piece_id=NSXCFVVUPVXPC75SQ2HZEFDSG7WYOOOGVWHSTE2GQLJ4AMW2DR7A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219531349 (375h29m59.990172684s) [noise=true , address=45.91.116.14:4190    , piece_id=3SNJWXBD6IM37H2OB3FMQPJNRXROQMKLCLREJNOXR4EVGFZ3EWHQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.219825963 (375h29m59.990467298s) [noise=true , address=45.91.116.10:4083    , piece_id=QB7IZNTRXUZUB2I7F3ENTX5VUMR2SQ4ZMXYX63OZ7BBVBF6OQNEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.223483851 (375h29m59.994125176s) [noise=true , address=45.91.116.4:4937     , piece_id=K32VK6T6D42USHFPN2ZKVNASBDVGFZIPY7ANGLDBUVXOPKTBNFYA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.233603090 (375h30m0.004244425s) [noise=true , address=45.91.116.10:4082    , piece_id=U4FCLI2EKP4BLSRVQIM4NDGP2RFTXCBMT52MES6AWXRFHNHBG3UA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.239221890 (375h30m0.009863225s) [noise=true , address=45.91.116.5:4299     , piece_id=R7W27CAHTJ2J4A7R5ED4MCKYTJFIK7ATKYSOFQQQ4UA7SKTR2L7Q, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.283055430 (375h30m0.053696765s) [noise=true , address=77.90.31.18:4805     , piece_id=MT357YKF45EP7DJGF6IVU5CXL34JKSAOJKVGDC6K4J2A3TL7GCPA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305071960 (375h30m0.075713295s) [noise=true , address=45.91.116.8:4045     , piece_id=BL5CCIPZ6CKYUUR472N2YTJIVG6YGSAU4C5TCVHZLS6NKMMP6HEQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.305351565 (375h30m0.0759929s) [noise=true , address=45.91.116.7:4010     , piece_id=3CZOPS4457RC53IRBK65O5HPKEKAONDNMVXO6EMLYZDC7VCQIXBQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:25.310763678 (375h30m0.081405013s) [noise=true , address=45.91.116.13:4153    , piece_id=QS25MZIB75FXI5NSQX727J4LO2O3TXLWOERNBFIZRGGXOKUJN4BA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:27.129473449 (375h30m1.900114784s) [noise=true , address=104.36.82.16:4294    , piece_id=QGZLYRKLPR6US4HR5LBYUPD2OFSKT4VE6WGOKLKAPCFWHC3QH2LA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:42.274686523 (375h30m17.045327858s) [noise=true , address=104.36.82.7:4910     , piece_id=E45U3F5CXZRUFHLKP67643QREDMFI54ZA4KHK4HT4VF5BXHTZTIA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: <nil>
Jul  9 00:40:52.188063012 (375h30m26.958704347s) [noise=true , address=104.36.82.14:4222    , piece_id=EZ65N3KNF7L4K7Y2SJIESME7I7ITAWS4IVMN2CSJS2MA26UENHKQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:40:52.188129919 (375h30m26.958771244s) [noise=true , address=104.36.82.2:4729     , piece_id=S4MWPIVNLQ3QVPTL3WZ7ASCBYBP7W3FJXWFANS52MNWLZW75CU3A, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35140->104.36.82.2:4729: use of closed network connection
Jul  9 00:40:52.188272542 (375h30m26.958913877s) [noise=true , address=104.36.82.14:4223    , piece_id=WVEVHZDJWVX6BNXU5JANS2EONIGDELN4A6O27EEAZM7HVAGIUTVQ, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57838->104.36.82.14:4223: use of closed network connection
Jul  9 00:40:52.188319602 (375h30m26.958960927s) [noise=true , address=104.36.82.9:4006     , piece_id=PTCVRETYYRJVDYD3P5GJTFTJP5FES5NVOLZOEXFP3JXJLE3QKEXA, seg_pos={0 0}, upload=300409]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49218->104.36.82.9:4006: use of closed network connection
Jul  9 00:40:52.188364118 (375h30m26.959005443s) [seg_pos={0 0}, upload=300409]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:40:52.422425287 (375h30m27.193066622s) [upload=300409]: Done uploading segments...
Jul  9 00:40:52.422450515 (375h30m27.19309184s) [upload=300409]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 25.507026133s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
35 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXXXXXXXX.X....X..XXXXXXX.XXXX.XXX...............]
duration: 
46.550894474s
FAIL (2025-07-09 00:46:20 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 46.55090781s core uplink us-select-1 us1-placement-20