Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 8.154928412s FAIL (2025-09-06 11:24:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 8.154954291s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 9.361676791s FAIL (2025-09-06 11:23:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 9.361701988s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 13.368133697s FAIL (2025-09-06 11:23:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 13.368164756s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.892436031s FAIL (2025-09-06 11:22:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.892469024s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 7.540445597s FAIL (2025-09-06 11:21:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 7.54049361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 7.026972381s FAIL (2025-09-06 11:21:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 7.027000035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 8.346319406s FAIL (2025-09-06 11:21:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 8.346339144s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.962044967s FAIL (2025-09-06 11:20:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 5.962077038s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.891300473s FAIL (2025-09-06 11:19:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.891327734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.348091998s FAIL (2025-09-06 11:18:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.348122497s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.033402653s FAIL (2025-09-06 11:17:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.033428842s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 7.861295116s FAIL (2025-09-06 11:16:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 7.861322578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 10.156098248s FAIL (2025-09-06 11:15:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 10.156126562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.144219373s FAIL (2025-09-06 11:14:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.144275151s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 11.129534741s FAIL (2025-09-06 11:14:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 11.129558857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 11.532508151s FAIL (2025-09-06 11:12:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 11.532534071s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.532831075s FAIL (2025-09-06 11:11:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.532858847s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 9.064197129s FAIL (2025-09-06 11:11:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 9.064223359s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 20.373109969s FAIL (2025-09-06 11:10:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 20.373133674s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.932094258s FAIL (2025-09-06 11:10:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.93212605s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.435364106s FAIL (2025-09-06 11:09:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 6.435393933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 17.65981223s FAIL (2025-09-06 11:08:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 17.659826848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 8.169535183s FAIL (2025-09-06 11:07:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 8.169570431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 11.707357444s FAIL (2025-09-06 11:07:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 11.707383835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002292982s FAIL (2025-09-06 11:06:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:06:12.764782499 (188h58m9.444978088s) [upload=166264]: Starting upload Sep 6 11:06:12.764915303 (188h58m9.445110862s) [upload=166264]: Done starting upload Sep 6 11:06:12.765165923 (188h58m9.445361492s) [upload=166264]: Uploading segments... Sep 6 11:06:12.765377528 (188h58m9.445573107s) [upload=166264]: Got next segment. Inline: false Sep 6 11:06:12.765423295 (188h58m9.445618864s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:06:12.765429828 (188h58m9.445625397s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:06:13.036787450 (188h58m9.716983019s) [seg_pos={0 0}, upload=166264]: Begin upload segment... Sep 6 11:06:13.036818079 (188h58m9.717013648s) [seg_pos={0 0}, upload=166264]: Done begin upload segment. Sep 6 11:06:13.036835643 (188h58m9.717031212s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.036858367 (188h58m9.717053936s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037155595 (188h58m9.717351164s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037364645 (188h58m9.717560214s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037412857 (188h58m9.717608426s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037697523 (188h58m9.717893092s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.037788196 (188h58m9.717983765s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038126693 (188h58m9.718322262s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038233117 (188h58m9.718428676s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038387212 (188h58m9.718582771s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038641578 (188h58m9.718837148s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038789121 (188h58m9.71898469s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.038895083 (188h58m9.719090652s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039111217 (188h58m9.719306786s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039172365 (188h58m9.719367934s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:13.039808371 (188h58m9.720003941s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Uploading piece... Sep 6 11:06:14.759985401 (188h58m11.44018097s) [upload=166264]: Next returned nil segment Sep 6 11:06:14.760000970 (188h58m11.440196539s) [upload=166264]: Waiting for error group managing segments... Sep 6 11:06:15.166892892 (188h58m11.847088461s) [noise=true , address=45.91.116.12:4226 , piece_id=COPIROBD4HJZUI2JFR4MJKYML7BDIJJTHJDLB7JIR6T2L2MIJW4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.175851365 (188h58m11.856046934s) [noise=true , address=45.91.116.10:4082 , piece_id=UUCK7ZYHC4VKFNPL73MQD7NFJG6XCJYCLBARRXLUIZD5AYRBRBGA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.176696382 (188h58m11.856891951s) [noise=true , address=45.91.116.12:4227 , piece_id=RP64I7RVEPLRG24NDNL6MIMJIHGFMC5PSRV5WOZ56K7L7F6NDXFQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.177687969 (188h58m11.857883538s) [noise=true , address=45.91.116.14:4190 , piece_id=GMMZUNC4T6RCCINXMWPAIZ2ESKW7T3AB3ROVHRMJONIHRBTXZL3Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.221317976 (188h58m11.901513545s) [noise=true , address=45.91.116.5:4299 , piece_id=FODO2EX7GGYD7DOFE3XUF7TLUPNCYXJB7UU3OD3ENHZYHC3LAG2A, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253471553 (188h58m11.933667122s) [noise=true , address=45.91.116.4:4938 , piece_id=FWEE4ZNNSLDNC4N4ZN4FUZK5T2TWHWKIFC7GBVAUKD7PCPMWPAZQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.253546587 (188h58m11.933742156s) [noise=true , address=77.90.31.13:4626 , piece_id=BYIAU4EQZU7Q6OGM5YSE5WIRPZPILJFPD4Y7QFJYJI33IKEOU6LQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.265198564 (188h58m11.945394133s) [noise=true , address=45.91.116.6:4975 , piece_id=SAUXLVRBL7QGZTX2JHLDN2WX4J7MOGOTBE6I4QAOFUPINB2BCODQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.283383982 (188h58m11.963579551s) [noise=true , address=45.91.116.11:4117 , piece_id=A7U5D3NF4ERDVYFQSGRMCUII7F6D4NYEG32CTFNDMU5XYCYASO4Q, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.305996572 (188h58m11.986192141s) [noise=true , address=77.90.31.19:4841 , piece_id=IAFCVOUWDXILDYWG3QPPRWEBM454YWIJFX5SI33P5SLMH52MEXKQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.315203380 (188h58m11.995398949s) [noise=true , address=77.90.31.18:4807 , piece_id=X44F2UIA24VCVQRA4UQKBVVWNUMKNP6ODWHH7JAOGSRW7YMBTNNQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.328211020 (188h58m12.008406589s) [noise=true , address=77.90.31.12:4592 , piece_id=NKSWKYBROLTRI5W2O3U3ISRLF7XZYREOOOUGVVMZS2ASV3J3KGRA, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342920246 (188h58m12.023115815s) [noise=true , address=45.91.116.11:4118 , piece_id=J2SLQW6P23MPANWITAKXDBR52FWI7CHYHZM4MDUCF4L5D7ZPEGEQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: <nil> Sep 6 11:06:15.342948540 (188h58m12.023144109s) [seg_pos={0 0}, upload=166264]: Segment reached optimal threshold of 13 pieces. Sep 6 11:06:15.343097476 (188h58m12.023293045s) [noise=true , address=77.90.31.19:4842 , piece_id=5HP3RTBPBYZ53VLULMRTR5JOH7YWRBCTSAREMQMOKCPN4OPCCLOQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343122604 (188h58m12.023318173s) [noise=true , address=77.90.31.14:4662 , piece_id=CABGG6HJG33NWJRTP66PIOTU2ZBKFAC43AV6IPF2VNMQVOIIK2NQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343184262 (188h58m12.023379831s) [noise=true , address=77.90.31.21:4878 , piece_id=RHK64XPQ5OZXUOK6LPK4KIMTNXM6ONGT2R27URKCCTCO6P56O6XQ, seg_pos={0 0}, upload=166264]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 11:06:15.343204290 (188h58m12.023399859s) [seg_pos={0 0}, upload=166264]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:06:15.343266519 (188h58m12.023462088s) [upload=166264]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:06:15.343270907 (188h58m12.023466476s) [upload=166264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:06:15.695036643 (188h58m12.375232212s) [upload=166264]: Done uploading segments... Sep 6 11:06:15.695056271 (188h58m12.37525184s) [upload=166264]: Upload finished. err: <nil> Duration: 30.002321337s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 3.261595769s FAIL (2025-09-06 10:37:21 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 3.261624765s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 3.208472766s FAIL (2025-09-06 10:37:06 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 3.208513104s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 3.790922578s FAIL (2025-09-06 10:36:49 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 3.790952894s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.099740141s FAIL (2025-09-06 10:36:29 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:48:21.207202480 (187h40m17.887398049s) [upload=165129]: Starting upload Sep 6 09:48:21.207262054 (187h40m17.887457623s) [upload=165129]: Done starting upload Sep 6 09:48:21.207314074 (187h40m17.887509643s) [upload=165129]: Uploading segments... Sep 6 09:48:21.207347838 (187h40m17.887543407s) [upload=165129]: Got next segment. Inline: true Sep 6 09:48:21.207351706 (187h40m17.887547275s) [upload=165129]: Waiting for error group managing segments... Sep 6 09:48:21.207382404 (187h40m17.887577973s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:48:21.207386171 (187h40m17.88758174s) [upload=165129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:48:21.207388706 (187h40m17.887584275s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:48:37.834981681 (187h40m34.51517725s) [upload=165129]: Done uploading segments... Sep 6 09:48:37.834999625 (187h40m34.515195194s) [upload=165129]: Upload finished. err: <nil> Duration:* 2.099765109s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 3.251288688s FAIL (2025-09-06 10:35:50 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 3.251320239s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 11.694066276s FAIL (2025-09-06 10:35:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 7N3JQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235101266 (187h40m33.915296835s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235220855 (187h40m33.915416424s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235421539 (187h40m33.915617108s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235433181 (187h40m33.91562875s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235644736 (187h40m33.915840305s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235709700 (187h40m33.915905269s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:49:05.524260987 (187h41m2.204456556s) [upload=165133]: Next segment err: context deadline exceeded Sep 6 09:49:05.525151190 (187h41m2.205346759s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525269517 (187h41m2.205465086s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection Sep 6 09:49:05.525307279 (187h41m2.205502848s) [noise=true , address=45.91.116.7:4011 , piece_id=JEKHNX25SUJ7XXI7WFJSABFK7C6GYSCX7O5WKID3ZVDZKBIUCM5Q, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection Sep 6 09:49:05.525345782 (187h41m2.205541351s) [noise=true , address=45.91.116.14:4189 , piece_id=GKEQ2IYP2FO2MFJGTXLQRUMA4BBM3A2TZYZLDCZ6GNJCNRFIKLBA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection Sep 6 09:49:05.525387101 (187h41m2.20558267s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection Sep 6 09:49:05.525432948 (187h41m2.205628517s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:58976->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525464129 (187h41m2.205659688s) [noise=true , address=45.91.116.5:4298 , piece_id=ACPADRRAY3XCMND27HLWYTHBYN6JYJHX65TGHOYX6KQLAQ4H2IEA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:37724->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525495419 (187h41m2.205690988s) [noise=true , address=45.91.116.10:4082 , piece_id=BHVWAODV7TRSZVHBHIB6WOZRUDMLH4PYSGWPFMZDO2YZTX5J4FCQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:34896->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525517281 (187h41m2.20571285s) [noise=true , address=45.91.116.5:4297 , piece_id=FM3W57L7DQXWDHRD45GB55M3EFWZC6F4F5MPOUND437L4VUHX5BA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525581042 (187h41m2.205776621s) [noise=true , address=45.91.116.14:4191 , piece_id=QSLHTOV3OHJ6ON4CPQOQFOWV3NW7MFVQ5R2G44O3VUBU3X3YEFWA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection Sep 6 09:49:05.525641939 (187h41m2.205837508s) [noise=true , address=45.91.116.13:4155 , piece_id=XV4N554TZTQFX5NE2LX6TRIGVG7TAXCZSQ7WCOQEK2WNZNWRVKLQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection Sep 6 09:49:05.525702134 (187h41m2.205897703s) [noise=true , address=45.91.116.6:4974 , piece_id=VF7QLD6YLKXRLBTNJ3C3JXTPVBM3TBGQNJNH47KJW7RWXVC5KW4A, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:44540->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525739145 (187h41m2.205934714s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:54696->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525773911 (187h41m2.20596948s) [noise=true , address=45.91.116.4:4938 , piece_id=4QW76PY2ZTXFMNTG75PVJGEEQG5OUZSTHRUOY4JB5N3NRNLVSSHA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:57880->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525780784 (187h41m2.205976353s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:51536->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525841049 (187h41m2.206036618s) [noise=true , address=45.91.116.6:4973 , piece_id=X7D4J4KTTIWBNVAOPMTMP3N6X6EE5VZLAGEZLWIWGLDBIUB7N3JQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:43034->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525861449 (187h41m2.206057018s) [seg_pos={0 0}, upload=165133]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 11.694090482s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.971355532s FAIL (2025-09-06 10:35:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:48:21.207202480 (187h40m17.887398049s) [upload=165129]: Starting upload Sep 6 09:48:21.207262054 (187h40m17.887457623s) [upload=165129]: Done starting upload Sep 6 09:48:21.207314074 (187h40m17.887509643s) [upload=165129]: Uploading segments... Sep 6 09:48:21.207347838 (187h40m17.887543407s) [upload=165129]: Got next segment. Inline: true Sep 6 09:48:21.207351706 (187h40m17.887547275s) [upload=165129]: Waiting for error group managing segments... Sep 6 09:48:21.207382404 (187h40m17.887577973s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:48:21.207386171 (187h40m17.88758174s) [upload=165129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:48:21.207388706 (187h40m17.887584275s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:48:37.834981681 (187h40m34.51517725s) [upload=165129]: Done uploading segments... Sep 6 09:48:37.834999625 (187h40m34.515195194s) [upload=165129]: Upload finished. err: <nil> Duration:* 1.971377674s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 3.368382942s FAIL (2025-09-06 10:34:57 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 3.368405825s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.142188588s FAIL (2025-09-06 10:34:52 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:48:21.207202480 (187h40m17.887398049s) [upload=165129]: Starting upload Sep 6 09:48:21.207262054 (187h40m17.887457623s) [upload=165129]: Done starting upload Sep 6 09:48:21.207314074 (187h40m17.887509643s) [upload=165129]: Uploading segments... Sep 6 09:48:21.207347838 (187h40m17.887543407s) [upload=165129]: Got next segment. Inline: true Sep 6 09:48:21.207351706 (187h40m17.887547275s) [upload=165129]: Waiting for error group managing segments... Sep 6 09:48:21.207382404 (187h40m17.887577973s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:48:21.207386171 (187h40m17.88758174s) [upload=165129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:48:21.207388706 (187h40m17.887584275s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:48:37.834981681 (187h40m34.51517725s) [upload=165129]: Done uploading segments... Sep 6 09:48:37.834999625 (187h40m34.515195194s) [upload=165129]: Upload finished. err: <nil> Duration:* 2.142216011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.321027402s FAIL (2025-09-06 10:34:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 7N3JQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235101266 (187h40m33.915296835s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235220855 (187h40m33.915416424s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235421539 (187h40m33.915617108s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235433181 (187h40m33.91562875s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235644736 (187h40m33.915840305s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235709700 (187h40m33.915905269s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:49:05.524260987 (187h41m2.204456556s) [upload=165133]: Next segment err: context deadline exceeded Sep 6 09:49:05.525151190 (187h41m2.205346759s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525269517 (187h41m2.205465086s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection Sep 6 09:49:05.525307279 (187h41m2.205502848s) [noise=true , address=45.91.116.7:4011 , piece_id=JEKHNX25SUJ7XXI7WFJSABFK7C6GYSCX7O5WKID3ZVDZKBIUCM5Q, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection Sep 6 09:49:05.525345782 (187h41m2.205541351s) [noise=true , address=45.91.116.14:4189 , piece_id=GKEQ2IYP2FO2MFJGTXLQRUMA4BBM3A2TZYZLDCZ6GNJCNRFIKLBA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection Sep 6 09:49:05.525387101 (187h41m2.20558267s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection Sep 6 09:49:05.525432948 (187h41m2.205628517s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:58976->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525464129 (187h41m2.205659688s) [noise=true , address=45.91.116.5:4298 , piece_id=ACPADRRAY3XCMND27HLWYTHBYN6JYJHX65TGHOYX6KQLAQ4H2IEA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:37724->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525495419 (187h41m2.205690988s) [noise=true , address=45.91.116.10:4082 , piece_id=BHVWAODV7TRSZVHBHIB6WOZRUDMLH4PYSGWPFMZDO2YZTX5J4FCQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:34896->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525517281 (187h41m2.20571285s) [noise=true , address=45.91.116.5:4297 , piece_id=FM3W57L7DQXWDHRD45GB55M3EFWZC6F4F5MPOUND437L4VUHX5BA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525581042 (187h41m2.205776621s) [noise=true , address=45.91.116.14:4191 , piece_id=QSLHTOV3OHJ6ON4CPQOQFOWV3NW7MFVQ5R2G44O3VUBU3X3YEFWA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection Sep 6 09:49:05.525641939 (187h41m2.205837508s) [noise=true , address=45.91.116.13:4155 , piece_id=XV4N554TZTQFX5NE2LX6TRIGVG7TAXCZSQ7WCOQEK2WNZNWRVKLQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection Sep 6 09:49:05.525702134 (187h41m2.205897703s) [noise=true , address=45.91.116.6:4974 , piece_id=VF7QLD6YLKXRLBTNJ3C3JXTPVBM3TBGQNJNH47KJW7RWXVC5KW4A, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:44540->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525739145 (187h41m2.205934714s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:54696->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525773911 (187h41m2.20596948s) [noise=true , address=45.91.116.4:4938 , piece_id=4QW76PY2ZTXFMNTG75PVJGEEQG5OUZSTHRUOY4JB5N3NRNLVSSHA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:57880->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525780784 (187h41m2.205976353s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:51536->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525841049 (187h41m2.206036618s) [noise=true , address=45.91.116.6:4973 , piece_id=X7D4J4KTTIWBNVAOPMTMP3N6X6EE5VZLAGEZLWIWGLDBIUB7N3JQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:43034->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525861449 (187h41m2.206057018s) [seg_pos={0 0}, upload=165133]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.3210521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 7.091286705s FAIL (2025-09-06 10:34:29 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 7.091329416s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 8.319479377s FAIL (2025-09-06 10:34:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 7N3JQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235101266 (187h40m33.915296835s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235220855 (187h40m33.915416424s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235421539 (187h40m33.915617108s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235433181 (187h40m33.91562875s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235644736 (187h40m33.915840305s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235709700 (187h40m33.915905269s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:49:05.524260987 (187h41m2.204456556s) [upload=165133]: Next segment err: context deadline exceeded Sep 6 09:49:05.525151190 (187h41m2.205346759s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525269517 (187h41m2.205465086s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection Sep 6 09:49:05.525307279 (187h41m2.205502848s) [noise=true , address=45.91.116.7:4011 , piece_id=JEKHNX25SUJ7XXI7WFJSABFK7C6GYSCX7O5WKID3ZVDZKBIUCM5Q, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection Sep 6 09:49:05.525345782 (187h41m2.205541351s) [noise=true , address=45.91.116.14:4189 , piece_id=GKEQ2IYP2FO2MFJGTXLQRUMA4BBM3A2TZYZLDCZ6GNJCNRFIKLBA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection Sep 6 09:49:05.525387101 (187h41m2.20558267s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection Sep 6 09:49:05.525432948 (187h41m2.205628517s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:58976->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525464129 (187h41m2.205659688s) [noise=true , address=45.91.116.5:4298 , piece_id=ACPADRRAY3XCMND27HLWYTHBYN6JYJHX65TGHOYX6KQLAQ4H2IEA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:37724->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525495419 (187h41m2.205690988s) [noise=true , address=45.91.116.10:4082 , piece_id=BHVWAODV7TRSZVHBHIB6WOZRUDMLH4PYSGWPFMZDO2YZTX5J4FCQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:34896->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525517281 (187h41m2.20571285s) [noise=true , address=45.91.116.5:4297 , piece_id=FM3W57L7DQXWDHRD45GB55M3EFWZC6F4F5MPOUND437L4VUHX5BA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525581042 (187h41m2.205776621s) [noise=true , address=45.91.116.14:4191 , piece_id=QSLHTOV3OHJ6ON4CPQOQFOWV3NW7MFVQ5R2G44O3VUBU3X3YEFWA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection Sep 6 09:49:05.525641939 (187h41m2.205837508s) [noise=true , address=45.91.116.13:4155 , piece_id=XV4N554TZTQFX5NE2LX6TRIGVG7TAXCZSQ7WCOQEK2WNZNWRVKLQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection Sep 6 09:49:05.525702134 (187h41m2.205897703s) [noise=true , address=45.91.116.6:4974 , piece_id=VF7QLD6YLKXRLBTNJ3C3JXTPVBM3TBGQNJNH47KJW7RWXVC5KW4A, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:44540->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525739145 (187h41m2.205934714s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:54696->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525773911 (187h41m2.20596948s) [noise=true , address=45.91.116.4:4938 , piece_id=4QW76PY2ZTXFMNTG75PVJGEEQG5OUZSTHRUOY4JB5N3NRNLVSSHA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:57880->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525780784 (187h41m2.205976353s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:51536->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525841049 (187h41m2.206036618s) [noise=true , address=45.91.116.6:4973 , piece_id=X7D4J4KTTIWBNVAOPMTMP3N6X6EE5VZLAGEZLWIWGLDBIUB7N3JQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:43034->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525861449 (187h41m2.206057018s) [seg_pos={0 0}, upload=165133]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 8.319505879s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.171176077s FAIL (2025-09-06 10:34:06 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:48:21.207202480 (187h40m17.887398049s) [upload=165129]: Starting upload Sep 6 09:48:21.207262054 (187h40m17.887457623s) [upload=165129]: Done starting upload Sep 6 09:48:21.207314074 (187h40m17.887509643s) [upload=165129]: Uploading segments... Sep 6 09:48:21.207347838 (187h40m17.887543407s) [upload=165129]: Got next segment. Inline: true Sep 6 09:48:21.207351706 (187h40m17.887547275s) [upload=165129]: Waiting for error group managing segments... Sep 6 09:48:21.207382404 (187h40m17.887577973s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:48:21.207386171 (187h40m17.88758174s) [upload=165129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:48:21.207388706 (187h40m17.887584275s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:48:37.834981681 (187h40m34.51517725s) [upload=165129]: Done uploading segments... Sep 6 09:48:37.834999625 (187h40m34.515195194s) [upload=165129]: Upload finished. err: <nil> Duration:* 2.17120312s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.128739034s FAIL (2025-09-06 10:33:51 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 45) Duration: 5.12876305s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.004780486s FAIL (2025-09-06 10:33:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230644309 (187h40m27.910839878s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230937310 (187h40m27.911132879s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231204531 (187h40m27.911400101s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231539021 (187h40m27.91173459s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231811022 (187h40m27.912006591s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:32.090388124 (187h40m28.770583693s) [upload=165131]: Next returned nil segment Sep 6 09:48:32.090408101 (187h40m28.77060367s) [upload=165131]: Waiting for error group managing segments... Sep 6 09:48:32.489381342 (187h40m29.169576911s) [noise=true , address=89.149.201.147:4014 , piece_id=NUXJQUIXFVVJUGUHIXQPYDLIZINQFSTMFGKVDL4O4RRKZNWQFUNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.501612347 (187h40m29.181807916s) [noise=true , address=89.149.201.147:4020 , piece_id=YA3VINMPBGDBIABG6ZJDDZUJSAEFJED3BFB2GK727WBIASJVMLXQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.512745170 (187h40m29.192940739s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.534695914 (187h40m29.214891483s) [noise=true , address=89.149.201.147:4022 , piece_id=GZPPXQCD7HQ5SWLK4CAYATYOGOCUVCCLARJTX7L7ALW7CKWADV4Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.537630178 (187h40m29.217825747s) [noise=true , address=89.149.201.147:4011 , piece_id=6ADPCUF2NQG4KXGOSMSMWRIAHWQSQ32VRTNQ5K7ZH4T3H6JQE2XQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.546466998 (187h40m29.226662567s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:18.452818084 (187h41m15.133013643s) [noise=true , address=185.167.204.205:4013 , piece_id=RV72BSLEVE3Q5ZFE5IWLTJREWRVBQQ5SNVFAJ5EJQAXABZPVIMGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:20.012900022 (187h41m16.693095591s) [noise=true , address=81.171.14.70:4045 , piece_id=FKHMXBLN4YMXA3E2IPIF652CDHCTFYRLDAPTYJ56QAAE2OVP45KQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.012709086 (187h41m17.692904655s) [noise=true , address=81.171.14.70:4047 , piece_id=77PDCGDOO5X32KEBFUYU2MFIRPA24V7XWRSSQVESKT4T3QNHBSLQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.626903100 (187h41m18.307098669s) [noise=true , address=185.167.204.205:4006 , piece_id=2HPGTJHD6ZVUPHMBYY2HODBU4FCP4M4NL3TN3VBW2UH37U7ZEF6A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.286253554 (187h41m18.966449123s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.805006087 (187h41m19.485201656s) [noise=true , address=81.171.14.70:4033 , piece_id=CLTHTQBLDHTWT4QY3SLFGVWLFWBAI64TWHXDWA46NUNTJJ5PRD2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.656975143 (187h41m20.337170712s) [noise=true , address=81.171.14.70:4041 , piece_id=R4JPY3MKUVY3HIQ2LKIGXCLCXQBME6DTYQNAWZLGOLQHCAE4OWNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.693921241 (187h41m20.37411681s) [noise=true , address=185.167.204.206:4063 , piece_id=6RD23BZ3WLATVOGIDMQU5B2HUISMBFGHA3VSZNC7H2B6C5VTKMVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.795040566 (187h41m20.475236135s) [noise=true , address=185.167.204.205:4009 , piece_id=HA6VCM2SK33OVWAP6YZY4FHSLEBPU6HQOEQNULY2LFIYDXMOWQUQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:24.328248240 (187h41m21.008443809s) [noise=true , address=81.171.14.70:4030 , piece_id=M6MLP7QCDQVOAN2YZX2I6GWJBTJ4BBC2OJKDFPNH5S7CPWREYA4A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.293846570 (187h41m21.974042139s) [noise=true , address=185.167.204.206:4038 , piece_id=CW6QOG56PYBJECWVJZUXVWR3ANS62ESBDK3PPAHPJDBEMKDT4PEQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.738447271 (187h41m22.41864284s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.308050849 (187h41m22.988246419s) [noise=true , address=185.167.204.205:4011 , piece_id=5LPZGI5FK266NV562CZDMQPYN4I24PFB65IIEYHPQIFLGQN7A7PQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.656713710 (187h41m23.336909279s) [noise=true , address=81.171.14.70:4026 , piece_id=3Q6LEURKQEBPYWHWA7ILZMJC3K7NPN4KK2OPPGXLSPB72PB7BHDA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:29.983835999 (187h41m26.664031568s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:30.384401717 (187h41m27.064597286s) [noise=true , address=81.171.14.70:4027 , piece_id=3SW3A6B4IESWTEXZNPN6BOKD3OJ4I76K22IXXPOLTB72EWXYUWWA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384648950 (187h41m27.064844519s) [noise=true , address=185.167.204.206:4062 , piece_id=YNECDFPNDKGGXHHCU2CWUPHYRAVMU7W5SUGYC2TANJAL4OT7MVVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384685079 (187h41m27.064880648s) [noise=true , address=185.167.204.205:4034 , piece_id=MV6C6QZNBNKW4XNQC5XUASI5WVDOSZIEDLS77NTCBTJFM7HB6UQQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384816821 (187h41m27.06501239s) [noise=true , address=185.167.204.206:4061 , piece_id=L6MCLBE2DW3INIOGRK3PWOJIEW6OAQAU72H74DFTILSAH5UBMS2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection Sep 6 09:49:30.384913305 (187h41m27.065108885s) [noise=true , address=185.167.204.205:4012 , piece_id=PQSOWQ5UQPOMHZJ4Q463MQ3ORQO2DABFW4AIJVLBFXC5376NY6SQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection Sep 6 09:49:30.384955877 (187h41m27.065151446s) [noise=true , address=185.167.204.205:4018 , piece_id=ALLU4AOERBQQ3G2PD2UZHUZWNYD5SKTKXX7B27VOONYUHUXZVSJA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection Sep 6 09:49:30.384988379 (187h41m27.065183948s) [noise=true , address=81.171.14.70:4031 , piece_id=JX373CUHJEWZFSHIODWS23ZP7IE7YOYQ4KC4YGRVNGSE2JX3W5BA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection Sep 6 09:49:30.385004431 (187h41m27.0652s) [seg_pos={0 0}, upload=165131]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Sep 6 09:49:30.560140329 (187h41m27.240335909s) [upload=165131]: Done uploading segments... Sep 6 09:49:30.560168635 (187h41m27.240364204s) [upload=165131]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.004813038s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.932586748s FAIL (2025-09-06 10:33:47 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 44) Duration: 2.932611083s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 6.071824488s FAIL (2025-09-06 10:33:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 7N3JQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235101266 (187h40m33.915296835s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235220855 (187h40m33.915416424s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235421539 (187h40m33.915617108s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235433181 (187h40m33.91562875s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235644736 (187h40m33.915840305s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235709700 (187h40m33.915905269s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:49:05.524260987 (187h41m2.204456556s) [upload=165133]: Next segment err: context deadline exceeded Sep 6 09:49:05.525151190 (187h41m2.205346759s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525269517 (187h41m2.205465086s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection Sep 6 09:49:05.525307279 (187h41m2.205502848s) [noise=true , address=45.91.116.7:4011 , piece_id=JEKHNX25SUJ7XXI7WFJSABFK7C6GYSCX7O5WKID3ZVDZKBIUCM5Q, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection Sep 6 09:49:05.525345782 (187h41m2.205541351s) [noise=true , address=45.91.116.14:4189 , piece_id=GKEQ2IYP2FO2MFJGTXLQRUMA4BBM3A2TZYZLDCZ6GNJCNRFIKLBA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection Sep 6 09:49:05.525387101 (187h41m2.20558267s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection Sep 6 09:49:05.525432948 (187h41m2.205628517s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:58976->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525464129 (187h41m2.205659688s) [noise=true , address=45.91.116.5:4298 , piece_id=ACPADRRAY3XCMND27HLWYTHBYN6JYJHX65TGHOYX6KQLAQ4H2IEA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:37724->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525495419 (187h41m2.205690988s) [noise=true , address=45.91.116.10:4082 , piece_id=BHVWAODV7TRSZVHBHIB6WOZRUDMLH4PYSGWPFMZDO2YZTX5J4FCQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:34896->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525517281 (187h41m2.20571285s) [noise=true , address=45.91.116.5:4297 , piece_id=FM3W57L7DQXWDHRD45GB55M3EFWZC6F4F5MPOUND437L4VUHX5BA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525581042 (187h41m2.205776621s) [noise=true , address=45.91.116.14:4191 , piece_id=QSLHTOV3OHJ6ON4CPQOQFOWV3NW7MFVQ5R2G44O3VUBU3X3YEFWA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection Sep 6 09:49:05.525641939 (187h41m2.205837508s) [noise=true , address=45.91.116.13:4155 , piece_id=XV4N554TZTQFX5NE2LX6TRIGVG7TAXCZSQ7WCOQEK2WNZNWRVKLQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection Sep 6 09:49:05.525702134 (187h41m2.205897703s) [noise=true , address=45.91.116.6:4974 , piece_id=VF7QLD6YLKXRLBTNJ3C3JXTPVBM3TBGQNJNH47KJW7RWXVC5KW4A, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:44540->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525739145 (187h41m2.205934714s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:54696->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525773911 (187h41m2.20596948s) [noise=true , address=45.91.116.4:4938 , piece_id=4QW76PY2ZTXFMNTG75PVJGEEQG5OUZSTHRUOY4JB5N3NRNLVSSHA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:57880->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525780784 (187h41m2.205976353s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:51536->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525841049 (187h41m2.206036618s) [noise=true , address=45.91.116.6:4973 , piece_id=X7D4J4KTTIWBNVAOPMTMP3N6X6EE5VZLAGEZLWIWGLDBIUB7N3JQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:43034->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525861449 (187h41m2.206057018s) [seg_pos={0 0}, upload=165133]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.071852292s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.034075775s FAIL (2025-09-06 10:33:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.034115311s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.480153804s FAIL (2025-09-06 10:33:32 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.48018246s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 4.99070238s FAIL (2025-09-06 10:33:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230644309 (187h40m27.910839878s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230937310 (187h40m27.911132879s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231204531 (187h40m27.911400101s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231539021 (187h40m27.91173459s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231811022 (187h40m27.912006591s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:32.090388124 (187h40m28.770583693s) [upload=165131]: Next returned nil segment Sep 6 09:48:32.090408101 (187h40m28.77060367s) [upload=165131]: Waiting for error group managing segments... Sep 6 09:48:32.489381342 (187h40m29.169576911s) [noise=true , address=89.149.201.147:4014 , piece_id=NUXJQUIXFVVJUGUHIXQPYDLIZINQFSTMFGKVDL4O4RRKZNWQFUNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.501612347 (187h40m29.181807916s) [noise=true , address=89.149.201.147:4020 , piece_id=YA3VINMPBGDBIABG6ZJDDZUJSAEFJED3BFB2GK727WBIASJVMLXQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.512745170 (187h40m29.192940739s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.534695914 (187h40m29.214891483s) [noise=true , address=89.149.201.147:4022 , piece_id=GZPPXQCD7HQ5SWLK4CAYATYOGOCUVCCLARJTX7L7ALW7CKWADV4Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.537630178 (187h40m29.217825747s) [noise=true , address=89.149.201.147:4011 , piece_id=6ADPCUF2NQG4KXGOSMSMWRIAHWQSQ32VRTNQ5K7ZH4T3H6JQE2XQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.546466998 (187h40m29.226662567s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:18.452818084 (187h41m15.133013643s) [noise=true , address=185.167.204.205:4013 , piece_id=RV72BSLEVE3Q5ZFE5IWLTJREWRVBQQ5SNVFAJ5EJQAXABZPVIMGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:20.012900022 (187h41m16.693095591s) [noise=true , address=81.171.14.70:4045 , piece_id=FKHMXBLN4YMXA3E2IPIF652CDHCTFYRLDAPTYJ56QAAE2OVP45KQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.012709086 (187h41m17.692904655s) [noise=true , address=81.171.14.70:4047 , piece_id=77PDCGDOO5X32KEBFUYU2MFIRPA24V7XWRSSQVESKT4T3QNHBSLQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.626903100 (187h41m18.307098669s) [noise=true , address=185.167.204.205:4006 , piece_id=2HPGTJHD6ZVUPHMBYY2HODBU4FCP4M4NL3TN3VBW2UH37U7ZEF6A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.286253554 (187h41m18.966449123s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.805006087 (187h41m19.485201656s) [noise=true , address=81.171.14.70:4033 , piece_id=CLTHTQBLDHTWT4QY3SLFGVWLFWBAI64TWHXDWA46NUNTJJ5PRD2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.656975143 (187h41m20.337170712s) [noise=true , address=81.171.14.70:4041 , piece_id=R4JPY3MKUVY3HIQ2LKIGXCLCXQBME6DTYQNAWZLGOLQHCAE4OWNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.693921241 (187h41m20.37411681s) [noise=true , address=185.167.204.206:4063 , piece_id=6RD23BZ3WLATVOGIDMQU5B2HUISMBFGHA3VSZNC7H2B6C5VTKMVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.795040566 (187h41m20.475236135s) [noise=true , address=185.167.204.205:4009 , piece_id=HA6VCM2SK33OVWAP6YZY4FHSLEBPU6HQOEQNULY2LFIYDXMOWQUQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:24.328248240 (187h41m21.008443809s) [noise=true , address=81.171.14.70:4030 , piece_id=M6MLP7QCDQVOAN2YZX2I6GWJBTJ4BBC2OJKDFPNH5S7CPWREYA4A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.293846570 (187h41m21.974042139s) [noise=true , address=185.167.204.206:4038 , piece_id=CW6QOG56PYBJECWVJZUXVWR3ANS62ESBDK3PPAHPJDBEMKDT4PEQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.738447271 (187h41m22.41864284s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.308050849 (187h41m22.988246419s) [noise=true , address=185.167.204.205:4011 , piece_id=5LPZGI5FK266NV562CZDMQPYN4I24PFB65IIEYHPQIFLGQN7A7PQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.656713710 (187h41m23.336909279s) [noise=true , address=81.171.14.70:4026 , piece_id=3Q6LEURKQEBPYWHWA7ILZMJC3K7NPN4KK2OPPGXLSPB72PB7BHDA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:29.983835999 (187h41m26.664031568s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:30.384401717 (187h41m27.064597286s) [noise=true , address=81.171.14.70:4027 , piece_id=3SW3A6B4IESWTEXZNPN6BOKD3OJ4I76K22IXXPOLTB72EWXYUWWA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384648950 (187h41m27.064844519s) [noise=true , address=185.167.204.206:4062 , piece_id=YNECDFPNDKGGXHHCU2CWUPHYRAVMU7W5SUGYC2TANJAL4OT7MVVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384685079 (187h41m27.064880648s) [noise=true , address=185.167.204.205:4034 , piece_id=MV6C6QZNBNKW4XNQC5XUASI5WVDOSZIEDLS77NTCBTJFM7HB6UQQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384816821 (187h41m27.06501239s) [noise=true , address=185.167.204.206:4061 , piece_id=L6MCLBE2DW3INIOGRK3PWOJIEW6OAQAU72H74DFTILSAH5UBMS2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection Sep 6 09:49:30.384913305 (187h41m27.065108885s) [noise=true , address=185.167.204.205:4012 , piece_id=PQSOWQ5UQPOMHZJ4Q463MQ3ORQO2DABFW4AIJVLBFXC5376NY6SQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection Sep 6 09:49:30.384955877 (187h41m27.065151446s) [noise=true , address=185.167.204.205:4018 , piece_id=ALLU4AOERBQQ3G2PD2UZHUZWNYD5SKTKXX7B27VOONYUHUXZVSJA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection Sep 6 09:49:30.384988379 (187h41m27.065183948s) [noise=true , address=81.171.14.70:4031 , piece_id=JX373CUHJEWZFSHIODWS23ZP7IE7YOYQ4KC4YGRVNGSE2JX3W5BA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection Sep 6 09:49:30.385004431 (187h41m27.0652s) [seg_pos={0 0}, upload=165131]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Sep 6 09:49:30.560140329 (187h41m27.240335909s) [upload=165131]: Done uploading segments... Sep 6 09:49:30.560168635 (187h41m27.240364204s) [upload=165131]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 4.990783103s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration: 2.145206461s FAIL (2025-09-06 10:33:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:48:21.207202480 (187h40m17.887398049s) [upload=165129]: Starting upload Sep 6 09:48:21.207262054 (187h40m17.887457623s) [upload=165129]: Done starting upload Sep 6 09:48:21.207314074 (187h40m17.887509643s) [upload=165129]: Uploading segments... Sep 6 09:48:21.207347838 (187h40m17.887543407s) [upload=165129]: Got next segment. Inline: true Sep 6 09:48:21.207351706 (187h40m17.887547275s) [upload=165129]: Waiting for error group managing segments... Sep 6 09:48:21.207382404 (187h40m17.887577973s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:48:21.207386171 (187h40m17.88758174s) [upload=165129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:48:21.207388706 (187h40m17.887584275s) [upload=165129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:48:37.834981681 (187h40m34.51517725s) [upload=165129]: Done uploading segments... Sep 6 09:48:37.834999625 (187h40m34.515195194s) [upload=165129]: Upload finished. err: <nil> Duration:* 2.145233633s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 4.959299668s FAIL (2025-09-06 10:33:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230644309 (187h40m27.910839878s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.230937310 (187h40m27.911132879s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231204531 (187h40m27.911400101s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231539021 (187h40m27.91173459s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:31.231811022 (187h40m27.912006591s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Uploading piece... Sep 6 09:48:32.090388124 (187h40m28.770583693s) [upload=165131]: Next returned nil segment Sep 6 09:48:32.090408101 (187h40m28.77060367s) [upload=165131]: Waiting for error group managing segments... Sep 6 09:48:32.489381342 (187h40m29.169576911s) [noise=true , address=89.149.201.147:4014 , piece_id=NUXJQUIXFVVJUGUHIXQPYDLIZINQFSTMFGKVDL4O4RRKZNWQFUNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.501612347 (187h40m29.181807916s) [noise=true , address=89.149.201.147:4020 , piece_id=YA3VINMPBGDBIABG6ZJDDZUJSAEFJED3BFB2GK727WBIASJVMLXQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.512745170 (187h40m29.192940739s) [noise=true , address=89.149.201.147:4017 , piece_id=ZSJJZSJPV77KT3JYWMUXGWBINY2HEACFO2YJGZE5C26QW6TIKFNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.534695914 (187h40m29.214891483s) [noise=true , address=89.149.201.147:4022 , piece_id=GZPPXQCD7HQ5SWLK4CAYATYOGOCUVCCLARJTX7L7ALW7CKWADV4Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.537630178 (187h40m29.217825747s) [noise=true , address=89.149.201.147:4011 , piece_id=6ADPCUF2NQG4KXGOSMSMWRIAHWQSQ32VRTNQ5K7ZH4T3H6JQE2XQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:48:32.546466998 (187h40m29.226662567s) [noise=true , address=89.149.201.147:4001 , piece_id=QRYXIE7K3ZZNWTMI5IVZ4XGLUKRJ633K3ETKUJ2SKTWNO5RCQ3GA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:18.452818084 (187h41m15.133013643s) [noise=true , address=185.167.204.205:4013 , piece_id=RV72BSLEVE3Q5ZFE5IWLTJREWRVBQQ5SNVFAJ5EJQAXABZPVIMGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:20.012900022 (187h41m16.693095591s) [noise=true , address=81.171.14.70:4045 , piece_id=FKHMXBLN4YMXA3E2IPIF652CDHCTFYRLDAPTYJ56QAAE2OVP45KQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.012709086 (187h41m17.692904655s) [noise=true , address=81.171.14.70:4047 , piece_id=77PDCGDOO5X32KEBFUYU2MFIRPA24V7XWRSSQVESKT4T3QNHBSLQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:21.626903100 (187h41m18.307098669s) [noise=true , address=185.167.204.205:4006 , piece_id=2HPGTJHD6ZVUPHMBYY2HODBU4FCP4M4NL3TN3VBW2UH37U7ZEF6A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.286253554 (187h41m18.966449123s) [noise=true , address=81.171.14.70:4036 , piece_id=J7JSNGNIR6FFKNYEXDGT2ONQAK6R73C3ITFWI3CTXCAWN6UI7RFQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:22.805006087 (187h41m19.485201656s) [noise=true , address=81.171.14.70:4033 , piece_id=CLTHTQBLDHTWT4QY3SLFGVWLFWBAI64TWHXDWA46NUNTJJ5PRD2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.656975143 (187h41m20.337170712s) [noise=true , address=81.171.14.70:4041 , piece_id=R4JPY3MKUVY3HIQ2LKIGXCLCXQBME6DTYQNAWZLGOLQHCAE4OWNQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.693921241 (187h41m20.37411681s) [noise=true , address=185.167.204.206:4063 , piece_id=6RD23BZ3WLATVOGIDMQU5B2HUISMBFGHA3VSZNC7H2B6C5VTKMVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:23.795040566 (187h41m20.475236135s) [noise=true , address=185.167.204.205:4009 , piece_id=HA6VCM2SK33OVWAP6YZY4FHSLEBPU6HQOEQNULY2LFIYDXMOWQUQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:24.328248240 (187h41m21.008443809s) [noise=true , address=81.171.14.70:4030 , piece_id=M6MLP7QCDQVOAN2YZX2I6GWJBTJ4BBC2OJKDFPNH5S7CPWREYA4A, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.293846570 (187h41m21.974042139s) [noise=true , address=185.167.204.206:4038 , piece_id=CW6QOG56PYBJECWVJZUXVWR3ANS62ESBDK3PPAHPJDBEMKDT4PEQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:25.738447271 (187h41m22.41864284s) [noise=true , address=185.167.204.205:4024 , piece_id=J35DFASGZNSAAAJJVXGLUKX4FI4L6XAHP4IVAAHIDOSZLPITL6PA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.308050849 (187h41m22.988246419s) [noise=true , address=185.167.204.205:4011 , piece_id=5LPZGI5FK266NV562CZDMQPYN4I24PFB65IIEYHPQIFLGQN7A7PQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:26.656713710 (187h41m23.336909279s) [noise=true , address=81.171.14.70:4026 , piece_id=3Q6LEURKQEBPYWHWA7ILZMJC3K7NPN4KK2OPPGXLSPB72PB7BHDA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:29.983835999 (187h41m26.664031568s) [noise=true , address=185.167.204.206:4048 , piece_id=W2QIC3CEQP5JNRY3OHEVBJL7JATEA5GLYTNX642XZQCB3VS3WNGQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: <nil> Sep 6 09:49:30.384401717 (187h41m27.064597286s) [noise=true , address=81.171.14.70:4027 , piece_id=3SW3A6B4IESWTEXZNPN6BOKD3OJ4I76K22IXXPOLTB72EWXYUWWA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384648950 (187h41m27.064844519s) [noise=true , address=185.167.204.206:4062 , piece_id=YNECDFPNDKGGXHHCU2CWUPHYRAVMU7W5SUGYC2TANJAL4OT7MVVA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384685079 (187h41m27.064880648s) [noise=true , address=185.167.204.205:4034 , piece_id=MV6C6QZNBNKW4XNQC5XUASI5WVDOSZIEDLS77NTCBTJFM7HB6UQQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:49:30.384816821 (187h41m27.06501239s) [noise=true , address=185.167.204.206:4061 , piece_id=L6MCLBE2DW3INIOGRK3PWOJIEW6OAQAU72H74DFTILSAH5UBMS2Q, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47066->185.167.204.206:4061: use of closed network connection Sep 6 09:49:30.384913305 (187h41m27.065108885s) [noise=true , address=185.167.204.205:4012 , piece_id=PQSOWQ5UQPOMHZJ4Q463MQ3ORQO2DABFW4AIJVLBFXC5376NY6SQ, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56056->185.167.204.205:4012: use of closed network connection Sep 6 09:49:30.384955877 (187h41m27.065151446s) [noise=true , address=185.167.204.205:4018 , piece_id=ALLU4AOERBQQ3G2PD2UZHUZWNYD5SKTKXX7B27VOONYUHUXZVSJA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54042->185.167.204.205:4018: use of closed network connection Sep 6 09:49:30.384988379 (187h41m27.065183948s) [noise=true , address=81.171.14.70:4031 , piece_id=JX373CUHJEWZFSHIODWS23ZP7IE7YOYQ4KC4YGRVNGSE2JX3W5BA, seg_pos={0 0}, upload=165131]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38592->81.171.14.70:4031: use of closed network connection Sep 6 09:49:30.385004431 (187h41m27.0652s) [seg_pos={0 0}, upload=165131]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Sep 6 09:49:30.560140329 (187h41m27.240335909s) [upload=165131]: Done uploading segments... Sep 6 09:49:30.560168635 (187h41m27.240364204s) [upload=165131]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 4.959317052s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 6.233515082s FAIL (2025-09-06 10:33:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 7N3JQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235101266 (187h40m33.915296835s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235220855 (187h40m33.915416424s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235421539 (187h40m33.915617108s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235433181 (187h40m33.91562875s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235644736 (187h40m33.915840305s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:48:37.235709700 (187h40m33.915905269s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Uploading piece... Sep 6 09:49:05.524260987 (187h41m2.204456556s) [upload=165133]: Next segment err: context deadline exceeded Sep 6 09:49:05.525151190 (187h41m2.205346759s) [noise=true , address=77.90.31.16:4734 , piece_id=EJCMCDAJ2MNO3TBGAESVUIRC6MFCMCE5RHOSL2VYAB2O3RNR2PVA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525269517 (187h41m2.205465086s) [noise=true , address=77.90.31.18:4806 , piece_id=FZGQ24A3E5W2WVD7Q22OIW54U77WZF57ZOXNEOYPOAOUY4ZXLYHQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33854->77.90.31.18:4806: use of closed network connection Sep 6 09:49:05.525307279 (187h41m2.205502848s) [noise=true , address=45.91.116.7:4011 , piece_id=JEKHNX25SUJ7XXI7WFJSABFK7C6GYSCX7O5WKID3ZVDZKBIUCM5Q, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37906->45.91.116.7:4011: use of closed network connection Sep 6 09:49:05.525345782 (187h41m2.205541351s) [noise=true , address=45.91.116.14:4189 , piece_id=GKEQ2IYP2FO2MFJGTXLQRUMA4BBM3A2TZYZLDCZ6GNJCNRFIKLBA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55912->45.91.116.14:4189: use of closed network connection Sep 6 09:49:05.525387101 (187h41m2.20558267s) [noise=true , address=45.91.116.6:4975 , piece_id=4B4E7OT5JVJBUNN4ZUZ5FKRC26BGZKBLKUTOZFDKO2IRDRCC3SDA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34404->45.91.116.6:4975: use of closed network connection Sep 6 09:49:05.525432948 (187h41m2.205628517s) [noise=true , address=45.91.116.8:4048 , piece_id=AFB2GS6PNU7P6JKSETSTX46KJPERVCBE4YBNKGRFGWD6K7HKIIXQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:58976->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525464129 (187h41m2.205659688s) [noise=true , address=45.91.116.5:4298 , piece_id=ACPADRRAY3XCMND27HLWYTHBYN6JYJHX65TGHOYX6KQLAQ4H2IEA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:37724->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525495419 (187h41m2.205690988s) [noise=true , address=45.91.116.10:4082 , piece_id=BHVWAODV7TRSZVHBHIB6WOZRUDMLH4PYSGWPFMZDO2YZTX5J4FCQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:34896->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525517281 (187h41m2.20571285s) [noise=true , address=45.91.116.5:4297 , piece_id=FM3W57L7DQXWDHRD45GB55M3EFWZC6F4F5MPOUND437L4VUHX5BA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525581042 (187h41m2.205776621s) [noise=true , address=45.91.116.14:4191 , piece_id=QSLHTOV3OHJ6ON4CPQOQFOWV3NW7MFVQ5R2G44O3VUBU3X3YEFWA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34852->45.91.116.14:4191: use of closed network connection Sep 6 09:49:05.525641939 (187h41m2.205837508s) [noise=true , address=45.91.116.13:4155 , piece_id=XV4N554TZTQFX5NE2LX6TRIGVG7TAXCZSQ7WCOQEK2WNZNWRVKLQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41822->45.91.116.13:4155: use of closed network connection Sep 6 09:49:05.525702134 (187h41m2.205897703s) [noise=true , address=45.91.116.6:4974 , piece_id=VF7QLD6YLKXRLBTNJ3C3JXTPVBM3TBGQNJNH47KJW7RWXVC5KW4A, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:44540->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525739145 (187h41m2.205934714s) [noise=true , address=45.91.116.11:4117 , piece_id=GVDJ2SDK2TSG2JE652PLZOSO2UV4K4VOWZ6UXH4BUG3LUWOMYWVQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:54696->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525773911 (187h41m2.20596948s) [noise=true , address=45.91.116.4:4938 , piece_id=4QW76PY2ZTXFMNTG75PVJGEEQG5OUZSTHRUOY4JB5N3NRNLVSSHA, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:57880->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525780784 (187h41m2.205976353s) [noise=true , address=45.91.116.10:4083 , piece_id=B76CYZG6IRYJVE5VW6WELIEJHBWT4GHXK3NMZET3J25KZLLAZZGQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:51536->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525841049 (187h41m2.206036618s) [noise=true , address=45.91.116.6:4973 , piece_id=X7D4J4KTTIWBNVAOPMTMP3N6X6EE5VZLAGEZLWIWGLDBIUB7N3JQ, seg_pos={0 0}, upload=165133]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:43034->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:49:05.525861449 (187h41m2.206057018s) [seg_pos={0 0}, upload=165133]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.233543757s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.180910247s FAIL (2025-09-06 10:33:01 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 45) Duration: 5.180940795s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 6.272367273s FAIL (2025-09-06 10:32:55 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.272394796s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)