Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX.X........................................................] duration: 5.868220903s FAIL (2025-07-19 06:02:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.868242605s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.688182532s FAIL (2025-07-19 06:02:28 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.68821304s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX.X.......................................................] duration: 5.969868907s FAIL (2025-07-19 06:01:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.969892492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.11323185s FAIL (2025-07-19 06:01:15 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 6.113258871s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX.X......................................................] duration: 5.958378147s FAIL (2025-07-19 06:00:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.958409628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.849550849s FAIL (2025-07-19 06:00:16 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.849580146s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XX.X.....................................................] duration: 5.677663285s FAIL (2025-07-19 05:59:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.677690385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.733124378s FAIL (2025-07-19 05:59:01 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.733152001s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XX.X....................................................] duration: 5.912375983s FAIL (2025-07-19 05:58:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.912401332s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XX.X...................................................] duration: 5.732569117s FAIL (2025-07-19 05:58:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.732599226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.617673198s FAIL (2025-07-19 05:57:50 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.617695771s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.....XX.X..................................................] duration: 5.931624496s FAIL (2025-07-19 05:57:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.931672407s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.735222815s FAIL (2025-07-19 05:56:47 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.735249496s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.....XX.X.................................................] duration: 5.912975267s FAIL (2025-07-19 05:56:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.913013059s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.608554451s FAIL (2025-07-19 05:55:58 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.6085804s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.800355606s FAIL (2025-07-19 05:54:50 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.800382658s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X.....XX.X................................................] duration: 5.770714453s FAIL (2025-07-19 05:54:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.770737076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 9.336224546s FAIL (2025-07-19 05:54:35 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 9.336259684s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.560714866s FAIL (2025-07-19 05:53:58 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.560738852s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.X.....XX.X...............................................] duration: 5.736367231s FAIL (2025-07-19 05:53:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.736390435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.859374749s FAIL (2025-07-19 05:53:38 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.85940157s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.676088422s FAIL (2025-07-19 05:53:05 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.676120665s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.X.X.....XX.X..............................................] duration: 5.73421385s FAIL (2025-07-19 05:52:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.734236765s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.791847734s FAIL (2025-07-19 05:51:59 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 5.791863124s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX.X.X.....XX.X.............................................] duration: 5.800073903s FAIL (2025-07-19 05:51:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.800101797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m10.522911743s FAIL (2025-07-19 05:51:53 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 2m10.522933855s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXX.X.X.....XX.X............................................] duration: 5.767136281s FAIL (2025-07-19 05:51:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.767164594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXX.X.X.....XX.X...........................................] duration: 5.721541614s FAIL (2025-07-19 05:50:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.721577302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXX.X.X.....XX.X..........................................] duration: 5.741178295s FAIL (2025-07-19 05:49:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.741201629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XXXXX.X.X.....XX.X.........................................] duration: 5.662361419s FAIL (2025-07-19 05:48:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.662390795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..XXXXX.X.X.....XX.X........................................] duration: 5.934503113s FAIL (2025-07-19 05:47:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.934528643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...XXXXX.X.X.....XX.X.......................................] duration: 5.94187423s FAIL (2025-07-19 05:46:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.941899869s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....XXXXX.X.X.....XX.X......................................] duration: 5.710136621s FAIL (2025-07-19 05:45:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.710161838s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X....XXXXX.X.X.....XX.X.....................................] duration: 5.743223638s FAIL (2025-07-19 05:44:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.743249948s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X....XXXXX.X.X.....XX.X....................................] duration: 5.824844067s FAIL (2025-07-19 05:43:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.824866972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X....XXXXX.X.X.....XX.X...................................] duration: 5.79904364s FAIL (2025-07-19 05:43:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.799071554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X..X....XXXXX.X.X.....XX.X..................................] duration: 5.784091001s FAIL (2025-07-19 05:42:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.784118774s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X..X....XXXXX.X.X.....XX.X.................................] duration: 5.81021241s FAIL (2025-07-19 05:41:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.810242968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X..X....XXXXX.X.X.....XX.X................................] duration: 5.737447946s FAIL (2025-07-19 05:41:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.737474216s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X..X....XXXXX.X.X.....XX.X...............................] duration: 5.691866233s FAIL (2025-07-19 05:41:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.691920735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X..X....XXXXX.X.X.....XX.X..............................] duration: 5.795256817s FAIL (2025-07-19 05:39:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.795282898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.509573329s FAIL (2025-07-19 05:39:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.509590853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.488925723s FAIL (2025-07-19 05:38:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.488952955s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X....X..X....XXXXX.X.X.....XX.X.............................] duration: 5.849740484s FAIL (2025-07-19 05:38:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.849778487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X....X..X....XXXXX.X.X.....XX.X............................] duration: 5.71405052s FAIL (2025-07-19 05:37:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.714072594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.470505797s FAIL (2025-07-19 05:37:36 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.470531185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.531982272s FAIL (2025-07-19 05:37:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.532008571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X....X..X....XXXXX.X.X.....XX.X...........................] duration: 5.718054054s FAIL (2025-07-19 05:36:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.718076869s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.513960048s FAIL (2025-07-19 05:36:32 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.513993071s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X....X..X....XXXXX.X.X.....XX.X..........................] duration: 5.720388227s FAIL (2025-07-19 05:36:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.720419097s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)