Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.980467668s FAIL (2025-10-12 07:53:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.980500129s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.891099474s FAIL (2025-10-12 07:53:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.891129443s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.704731497s FAIL (2025-10-12 07:52:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.704766704s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.866114567s FAIL (2025-10-12 07:51:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.86615263s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.909906297s FAIL (2025-10-12 07:51:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.909939279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.855126116s FAIL (2025-10-12 07:51:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.855158929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.815663882s FAIL (2025-10-12 07:50:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.815717304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.91646177s FAIL (2025-10-12 07:49:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.916496405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.154334993s FAIL (2025-10-12 07:48:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.154377935s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.853559702s FAIL (2025-10-12 07:47:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.853593016s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.146730752s FAIL (2025-10-12 07:46:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.146763185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.074144765s FAIL (2025-10-12 07:46:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.074174192s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.828117235s FAIL (2025-10-12 07:46:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.828158042s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.995928194s FAIL (2025-10-12 07:45:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.995959013s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.853254589s FAIL (2025-10-12 07:44:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.853282975s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.057840175s FAIL (2025-10-12 07:43:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.057863561s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.035509192s FAIL (2025-10-12 07:43:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.035543549s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.675402849s FAIL (2025-10-12 07:42:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.675445709s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.883211072s FAIL (2025-10-12 07:40:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.883243854s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.707332969s FAIL (2025-10-12 07:39:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.707409356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.844536759s FAIL (2025-10-12 07:38:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.844569501s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.356843078s FAIL (2025-10-12 07:38:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.356890377s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.057804391s FAIL (2025-10-12 07:36:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.05784082s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.925085237s FAIL (2025-10-12 07:36:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.925120514s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.71260126s FAIL (2025-10-12 07:35:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.712629534s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.931636922s FAIL (2025-10-12 07:34:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.9316773s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.041107425s FAIL (2025-10-12 07:33:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.041135928s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.536172324s FAIL (2025-10-12 07:33:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.536208704s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.024280213s FAIL (2025-10-12 07:32:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.024339605s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.023694957s FAIL (2025-10-12 07:31:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.023729693s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.971320928s FAIL (2025-10-12 07:31:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.971341587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m32.360048091s FAIL (2025-10-12 07:31:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 77730639s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416237166 (449h57m36.878329649s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416461945 (449h57m36.878554428s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.416916230 (449h57m36.879008713s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.417375644 (449h57m36.879468127s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418400055 (449h57m36.880492528s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418866062 (449h57m36.880958535s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.418979619 (449h57m36.881072102s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419378027 (449h57m36.88147051s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.419722002 (449h57m36.881814475s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420558705 (449h57m36.882651188s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.420895536 (449h57m36.882988009s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.421204444 (449h57m36.883296917s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.422858264 (449h57m36.884950747s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.423521166 (449h57m36.885613649s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424082044 (449h57m36.886174527s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:51.424779732 (449h57m36.886872215s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Uploading piece... Oct 12 07:28:52.786636217 (449h57m38.248728701s) [upload=395866]: Next returned nil segment Oct 12 07:28:52.786658098 (449h57m38.248750582s) [upload=395866]: Waiting for error group managing segments... Oct 12 07:28:53.125949247 (449h57m38.58804172s) [noise=true , address=64.49.13.139:4478 , piece_id=HVWUJRHVDFVXOXACMLC6S45IETYUIGYJIRUOKPAPBFB24AJMV6KA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.193237435 (449h57m38.655329908s) [noise=true , address=64.49.13.138:4487 , piece_id=U33EDVLL4EWXXYKHJORKQT5IPITCRHTSHX5O3XPRT7XNBOUVC2ZA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.281648515 (449h57m38.743740988s) [noise=true , address=145.79.227.52:4465 , piece_id=4I4HG5NO3ENHIUHNATI76ZJJ7H3TN6ZN3WBHTCOQEEZNUJUOEQSA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.290618025 (449h57m38.752710528s) [noise=true , address=145.79.227.49:4367 , piece_id=TOIU4HYKJQC5RJUMAVI7C3XR6KXU3YL776CWN6JXYWJ4P5WX65KQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.295090571 (449h57m38.757183044s) [noise=true , address=145.79.227.35:4499 , piece_id=6OBKKQN4QVX7KRGUKWLVFFFD5D5FUJ63T43TBVG2VJ3IS2AOZIDQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.407237007 (449h57m38.86932949s) [noise=true , address=45.91.116.13:4173 , piece_id=F55RKFNMQ4DP3XKTYDVHA7GNCFBJLXJLKWKG7SAKKXUYDJ2EFTOQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.410098435 (449h57m38.872190918s) [noise=true , address=45.91.116.8:4050 , piece_id=O4PTDNWNEOR4XJRWB6JURLIT73DCRC6WMEO4TWE2RLVRJNE445AQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.435342173 (449h57m38.897434646s) [noise=true , address=77.90.31.12:4605 , piece_id=6II3C4XJTZYCQMBLY5MVRFLKLHTIPOODQLYNI4F626BGHOZOAEBA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.446302454 (449h57m38.908394967s) [noise=true , address=45.91.116.7:4031 , piece_id=E6O2L7EQAFBKEKVE7IQBUZRGWNQI4DHBUWYAKHLAQ635GZ6JUTUA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.449744248 (449h57m38.911836731s) [noise=true , address=45.91.116.6:4995 , piece_id=ZV53CRRZMTP7L4LABA2Z6PVNPDJMOMVFEFAI3CWMS5K35RZ3CYIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.468040638 (449h57m38.930133111s) [noise=true , address=77.90.31.10:4540 , piece_id=CWKJYN2J46XP6BZIHR7CA72JOGL4E4CDUDDLP3676GKRNZWXFGIA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.473743357 (449h57m38.93583583s) [noise=true , address=45.91.116.12:4242 , piece_id=QCNPWTUODSXMSOIZC76NFPVNTINHLFO5YIECW6APBLWM5TSHG45A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.480387758 (449h57m38.942480232s) [noise=true , address=77.90.31.17:4801 , piece_id=5S6JFFEKPI24LBUAI74GSC2CEGBUFMVRYPZS5ISGVSGWCANQLOPA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512244629 (449h57m38.974337102s) [noise=true , address=45.91.116.10:4102 , piece_id=OYXOG34E7T5RTE6PEL4QFDR5JRY46STZ3PBQYJS3C6MCO5P7SGTA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: <nil> Oct 12 07:28:53.512311767 (449h57m38.97440424s) [seg_pos={0 0}, upload=395866]: Segment reached optimal threshold of 14 pieces. Oct 12 07:28:53.512475328 (449h57m38.974567812s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512483894 (449h57m38.974576378s) [noise=true , address=77.90.31.11:4575 , piece_id=BF3VNSBOZSXRJSK2DK4QB3KY73P3IZIKGMCU2BGYLO7SNXNANUOA, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512591209 (449h57m38.974683692s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xoAbv6zJuG9LjyCdsCrTWQaCDyLrWyv8RCrMTiqNqazoEwD1M): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512612971 (449h57m38.974705454s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512621867 (449h57m38.97471434s) [noise=true , address=77.90.31.14:4664 , piece_id=T3LLQNMNZCECGESCMXYXAHIAK67LONR6VMYP6RKKGHK4MD6Q2ITQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512646545 (449h57m38.974739028s) [noise=true , address=77.90.31.15:4727 , piece_id=NAPUST6O53HNISWR6CSGX2L2VSZFYYO5P7N2HNEQMIZB7PH4T52A, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512703072 (449h57m38.974795545s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4NHwWuUoNH51rQCv9jdJ4oKKHxMQJ2ZZJfrSZLWYWJJWipWK): protocol: expected piece hash; context canceled; EOF Oct 12 07:28:53.512709154 (449h57m38.974801637s) [noise=true , address=77.90.31.16:4767 , piece_id=RKNWM4BZ7Q4NE7CMKUI3AAF5DF3SPIZUYF3USU7Z36TAXSZDPNJQ, seg_pos={0 0}, upload=395866]: Long tail cancellation due to optimal threshold Oct 12 07:28:53.512738749 (449h57m38.974831233s) [seg_pos={0 0}, upload=395866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 12 07:28:53.512862896 (449h57m38.974955369s) [upload=395866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 12 07:28:53.512871603 (449h57m38.974964086s) [upload=395866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 12 07:31:22.820504910 (450h0m8.282597403s) [upload=395866]: Done uploading segments... Oct 12 07:31:22.820535368 (450h0m8.282627851s) [upload=395866]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m32.360075033s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.901812824s FAIL (2025-10-12 07:12:03 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.901839276s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.353102403s FAIL (2025-10-12 07:11:14 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.353125757s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.227023236s FAIL (2025-10-12 07:10:05 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.227059555s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.226141236s FAIL (2025-10-12 07:09:11 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.226174999s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.153784065s FAIL (2025-10-12 07:08:27 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.153815155s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.001151267s FAIL (2025-10-12 07:07:29 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.001183779s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.037291353s FAIL (2025-10-12 07:06:41 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.037327133s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.151346544s FAIL (2025-10-12 07:05:50 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.151367525s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.693138811s FAIL (2025-10-12 07:05:15 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.693172776s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.916452838s FAIL (2025-10-12 07:04:23 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.916487074s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.320745178s FAIL (2025-10-12 07:03:32 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.320782932s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.196099502s FAIL (2025-10-12 07:02:26 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.196170217s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.239023554s FAIL (2025-10-12 07:01:35 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.239062558s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.008217404s FAIL (2025-10-12 07:00:55 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.008250427s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.085803525s FAIL (2025-10-12 07:00:28 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.085874261s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.985096766s FAIL (2025-10-12 06:59:36 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.985134317s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.863410779s FAIL (2025-10-12 06:58:29 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.863442728s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.167905383s FAIL (2025-10-12 06:58:00 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.168005052s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)