Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.303967649s trace: https://jaeger.dev.storj.io/trace/24a1909cc1b5f82c FAIL (2026-05-19 07:36:13 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.303992946s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.040571468s trace: https://jaeger.dev.storj.io/trace/24a1909cc1b5f82c FAIL (2026-05-19 07:35:54 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.040596003s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.146728186s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:35:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.14676204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.253033887s trace: https://jaeger.dev.storj.io/trace/24a1909cc1b5f82c FAIL (2026-05-19 07:35:12 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.253050569s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.003758391s trace: https://jaeger.dev.storj.io/trace/24a1909cc1b5f82c FAIL (2026-05-19 07:35:07 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 30.003780243s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.717658809s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:34:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.717680461s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.601405065s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:34:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.601435283s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.430526965s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:33:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.430549769s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.327036749s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:32:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.32705343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 25.341865719s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:32:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 25.341889334s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.229033178s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:31:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.229057534s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.332675186s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:30:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.332704181s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 29.735570173s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:29:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 29.735601702s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.395906534s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:28:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.395923367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.564763847s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:28:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.564779066s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.734000213s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:28:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.734042233s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.608059422s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:27:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.60808413s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.301137276s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:26:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.301158896s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.208643874s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:25:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.208667068s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.303139603s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:24:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.303159831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.058647558s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:23:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.058663087s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.888506658s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:23:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.888520995s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 10.787294981s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:22:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 10.787322505s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.066692508s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:21:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.06673536s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 13.491991398s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:20:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 13.4920182s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.849464626s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:19:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.849491026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.711124173s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:18:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.711150733s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.372250588s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:17:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.372274694s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.780805792s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:16:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.78084156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.815818452s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:16:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.815862806s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.776660351s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:15:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.776683526s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.708431408s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:14:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.70845846s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.810084949s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:14:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 5.810109997s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.455484338s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:12:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.455506569s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.31075923s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:12:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 7.310786031s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 10.226745267s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:12:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 10.226759354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 10.209932936s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:10:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 10.20995572s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.07235934s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:09:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.072377535s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.325911817s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:08:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 6.325937005s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 8.059761384s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:08:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 8.059787004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 9.958189903s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:07:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 9.95821333s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.05361549s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:06:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 7.053631591s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 29.488738439s trace: https://jaeger.dev.storj.io/trace/2f9c89da71fb2653 FAIL (2026-05-19 07:06:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== May 19 07:06:17.868412334 (444h18m28.389236079s) [upload=426069]: Starting upload May 19 07:06:17.868501894 (444h18m28.38932564s) [upload=426069]: Done starting upload May 19 07:06:17.868575464 (444h18m28.389399209s) [upload=426069]: Uploading segments... May 19 07:06:17.868675946 (444h18m28.389499681s) [upload=426069]: Got next segment. Inline: false May 19 07:06:17.868701916 (444h18m28.389525651s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 19 07:06:17.868705703 (444h18m28.389529448s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 19 07:06:18.103595758 (444h18m28.624419553s) [seg_pos={0 0}, upload=426069]: Begin upload segment... May 19 07:06:18.103684477 (444h18m28.624508232s) [seg_pos={0 0}, upload=426069]: Done begin upload segment. May 19 07:06:18.103704675 (444h18m28.62452846s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103723541 (444h18m28.624547286s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.103799176 (444h18m28.624622921s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.104569415 (444h18m28.62539317s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105203325 (444h18m28.62602709s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.105707537 (444h18m28.626531292s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106188996 (444h18m28.627012751s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.106619958 (444h18m28.627443713s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107018649 (444h18m28.627842394s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107392162 (444h18m28.628215907s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.107826100 (444h18m28.628649845s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108249318 (444h18m28.629073063s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.108672696 (444h18m28.629496441s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.109100352 (444h18m28.629924097s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.110467149 (444h18m28.631290894s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:18.113835036 (444h18m28.634658781s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Uploading piece... May 19 07:06:19.407046757 (444h18m29.927870502s) [upload=426069]: Next returned nil segment May 19 07:06:19.407061995 (444h18m29.92788575s) [upload=426069]: Waiting for error group managing segments... May 19 07:06:19.743341599 (444h18m30.264165344s) [noise=true , address=193.149.164.78:4493 , piece_id=XMDDEPLGBL77VWLZWWWJQV74EYSYIM2NRMYBVXUO2CRJYDLXOMOQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.809130538 (444h18m30.329954283s) [noise=true , address=167.17.72.71:4789 , piece_id=HT53M3PLV5AFA3NVSS4ANMVIMNZLN2LOSNAGUP4ZPAS6UDSRMSAQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.811837964 (444h18m30.332661709s) [noise=true , address=64.49.13.27:4968 , piece_id=VRMPLOPTOBJ3WHVBAF4GYHC6PJMSX44GGFWD2LG7BWZWTMGBQVUQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.827823707 (444h18m30.348647452s) [noise=true , address=167.17.72.69:4732 , piece_id=XVNBSPM7PG7PEXOVY6XXDTVYY4YU75FQD46HMLEXVYAXYP62YNLQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.939936329 (444h18m30.460760074s) [noise=true , address=167.17.72.68:4648 , piece_id=DAGXKC7IU3HLGM2VDHPLM76VTE37YALW5GNWCFDRK6VUTOFEXA2A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.987088334 (444h18m30.507912079s) [noise=true , address=145.79.227.29:4956 , piece_id=DTTABHU2RE37GC6FH74BL34ZKVFXEGPMBSXBMERCIAUB4GZVILJQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:19.999172874 (444h18m30.519996629s) [noise=true , address=145.79.227.19:4089 , piece_id=DFCZKDFUNGP4LFPNY3NFKYINGVIETGLDQHHDMQIKJPQDAI6P72PA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.003854748 (444h18m30.524678513s) [noise=true , address=145.79.227.5:4229 , piece_id=TS4G46IQVLFO2DZEYU2CX3QTHLVMYAPXYEWOFGRVIYNZKCLGVRLA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.023342103 (444h18m30.544165848s) [noise=true , address=145.79.227.6:4272 , piece_id=JIS3XOECJYPZ5RZD4B6UGFTPC4FRDNMK755TZDJQLZDJ7WCXNVEA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.053126598 (444h18m30.573950343s) [noise=true , address=145.79.227.21:4487 , piece_id=4OPEESLEJGUYPTLA6BGNVKEI4TOARYAFC3AQZ3XLICBGJFBQMA6A, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:20.300112894 (444h18m30.820936639s) [noise=true , address=193.149.164.67:4005 , piece_id=IJEIHGG2WZWRO2KTMOIBOVHF6I77ESHVTFPLDKBQ2K3KBIEGLJ7Q, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:21.904360007 (444h18m32.425183752s) [noise=true , address=45.91.116.10:4111 , piece_id=2QI2UFV2ILTCIIAMVNCJ6M5RY5FFNGUCIGJYLE4H5OTND6RTQMTA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.219801165 (444h18m33.74062491s) [noise=true , address=45.91.116.13:4184 , piece_id=F6NE7PAPBZBXRGL2ISVUYW6INISN37EI2QVDZI7V62BTYSQF4RPQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457213703 (444h18m33.978037438s) [noise=true , address=45.91.116.6:4003 , piece_id=LLPADLSGB2ITI2N6KJIQCGTBKYMOPXNMQIM4CKS5YWJNO6M3V5DA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: <nil> May 19 07:06:23.457232960 (444h18m33.978056705s) [seg_pos={0 0}, upload=426069]: Satisfied cohort requirements after 14 pieces May 19 07:06:23.457370442 (444h18m33.978194187s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457378256 (444h18m33.978202001s) [noise=true , address=45.91.116.11:4122 , piece_id=PLUAQTIYUIPS756ASKYE5N2QARV2CVMOZUNNIN36SXSAGOPNV5LA, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457462808 (444h18m33.978286543s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3): protocol: expected piece hash; context canceled; EOF May 19 07:06:23.457467016 (444h18m33.978290761s) [noise=true , address=45.91.116.5:4309 , piece_id=D67UIM6KNXWA3ULLXFOVCNBFFOM5BBKV4YMLPO4X5ZMOKWCPEDBQ, seg_pos={0 0}, upload=426069]: Long tail cancellation due to optimal threshold May 19 07:06:23.457482957 (444h18m33.978306702s) [seg_pos={0 0}, upload=426069]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> May 19 07:06:23.457540466 (444h18m33.978364201s) [upload=426069]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 19 07:06:23.457544193 (444h18m33.978367928s) [upload=426069]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 19 07:06:23.736828300 (444h18m34.257652045s) [upload=426069]: Done uploading segments... May 19 07:06:23.736847335 (444h18m34.25767108s) [upload=426069]: Upload finished. err: <nil> Duration: 29.4887707s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 57.147626ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:11:33 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 57.171743ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 60.293779ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:11:10 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 60.316893ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 55.81749ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:10:44 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 55.840773ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 63.320385ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:09:54 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 63.346354ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 62.073244ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:09:35 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 62.092682ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 73.936863ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:08:41 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 73.95154ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 69.895641ms remote: "tcp:34.120.119.150:443" response headers: Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=298 Date=Tue, 19 May 2026 06:01:51 GMT Alt-Svc=h3=":443"; ma=2592000 target: https://www.tardigrade.io/ trace: https://jaeger.dev.storj.io/trace/66ea192edffbca9d FAIL (2026-05-19 06:07:39 UTC) orbiter warning: unexpected status code "503 Service Unavailable" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>503 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The service you requested is not available at this time.<p>Service error -27.</h2>\n<h2></h2>\n</body></html>\n" Duration: 69.922183ms core website

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)