Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.865468985s FAIL (2025-10-27 06:23:19 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.865497629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 7.985607142s FAIL (2025-10-27 06:23:14 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.985639925s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 809.908203ms target: https://us1.storj.io/login FAIL (2025-10-27 06:23:14 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 809.965572ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.919736346s FAIL (2025-10-27 06:23:04 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.91976517s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 624.492923ms target: https://version.storj.io/ FAIL (2025-10-27 06:23:02 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout Duration: 624.516197ms core website
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 795.483554ms target: https://us1.storj.io/login FAIL (2025-10-27 06:22:52 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 795.51816ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.004105373s FAIL (2025-10-27 06:22:40 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.004139036s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000956237s target: https://us1.storj.io/login FAIL (2025-10-27 06:22:40 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 30.000977918s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.553346943s FAIL (2025-10-27 06:22:39 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.553373563s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.282515482s FAIL (2025-10-27 06:22:34 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.282547103s core uplink us-select-1
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001191153s target: https://version.storj.io/ FAIL (2025-10-27 06:22:30 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout Duration: 30.001224537s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000978734s FAIL (2025-10-27 06:22:28 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1m0.001000677s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000955391s target: https://us1.storj.io/login FAIL (2025-10-27 06:22:10 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 30.000989958s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000679203s FAIL (2025-10-27 06:21:59 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 30.000702088s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000932478s FAIL (2025-10-27 06:21:57 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.000956233s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.862235714s FAIL (2025-10-27 05:49:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.862267366s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.931171473s FAIL (2025-10-27 05:48:29 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.93120166s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.893923912s FAIL (2025-10-27 05:47:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.893957436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.06389904s FAIL (2025-10-27 05:46:55 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.063931009s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.958807372s FAIL (2025-10-27 05:46:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.958841889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.879600698s FAIL (2025-10-27 05:45:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.879634011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.966343843s FAIL (2025-10-27 05:45:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.966385872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.959234669s FAIL (2025-10-27 05:44:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.959501859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.975592089s FAIL (2025-10-27 05:43:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.975627217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.92324208s FAIL (2025-10-27 05:42:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.923272007s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.864320002s FAIL (2025-10-27 05:42:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.864349278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.01776853s FAIL (2025-10-27 05:42:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.017803426s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.036461074s FAIL (2025-10-27 05:41:52 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.036484178s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.875008324s FAIL (2025-10-27 05:41:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.875039322s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.14563428s FAIL (2025-10-27 05:39:53 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.145665778s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.889191183s FAIL (2025-10-27 05:38:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.889214037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.228844869s FAIL (2025-10-27 05:37:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.22887163s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.111707418s FAIL (2025-10-27 05:37:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.111748625s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.943324231s FAIL (2025-10-27 05:36:52 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.943356202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.322918106s FAIL (2025-10-27 05:35:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.322949076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.924193996s FAIL (2025-10-27 05:34:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.924224965s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.88732551s FAIL (2025-10-27 05:34:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.887351709s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.980527053s FAIL (2025-10-27 05:33:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.980559615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.030974178s FAIL (2025-10-27 05:32:55 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.030999376s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.113107297s FAIL (2025-10-27 05:31:53 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.113141103s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.107353336s FAIL (2025-10-27 05:30:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.107391919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.0910271s FAIL (2025-10-27 05:30:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.091063408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.011031473s FAIL (2025-10-27 05:29:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.011064116s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.252736913s FAIL (2025-10-27 05:29:14 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.252784915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.080359043s FAIL (2025-10-27 05:28:19 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.080394218s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.186722279s FAIL (2025-10-27 05:27:21 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.186757375s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.92865986s FAIL (2025-10-27 05:26:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.928700026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.912380967s FAIL (2025-10-27 05:25:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.912410944s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.071627912s FAIL (2025-10-27 05:25:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 6.071669372s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.940396739s FAIL (2025-10-27 05:24:53 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 27 05:05:39.614505602 (807h34m25.076598085s) [upload=710322]: Starting upload Oct 27 05:05:39.614590723 (807h34m25.076683206s) [upload=710322]: Done starting upload Oct 27 05:05:39.614801895 (807h34m25.076894379s) [upload=710322]: Uploading segments... Oct 27 05:05:39.614884553 (807h34m25.076977027s) [upload=710322]: Got next segment. Inline: false Oct 27 05:05:39.614919851 (807h34m25.077012324s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 27 05:05:39.614924429 (807h34m25.077016902s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 27 05:05:39.871912696 (807h34m25.334005179s) [seg_pos={0 0}, upload=710322]: Begin upload segment... Oct 27 05:05:39.871980174 (807h34m25.334072647s) [seg_pos={0 0}, upload=710322]: Done begin upload segment. Oct 27 05:05:39.872017716 (807h34m25.334110189s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872035730 (807h34m25.334128213s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872136652 (807h34m25.334229125s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.872983715 (807h34m25.335076199s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874708670 (807h34m25.336801153s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.874794383 (807h34m25.336886866s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875705238 (807h34m25.337797721s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875726979 (807h34m25.337819462s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.875796682 (807h34m25.337889165s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.876982260 (807h34m25.339074753s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878682606 (807h34m25.34077509s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.878872058 (807h34m25.340964541s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.879915425 (807h34m25.342007908s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880286351 (807h34m25.342378834s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.880683337 (807h34m25.34277583s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:39.881915322 (807h34m25.344007815s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Uploading piece... Oct 27 05:05:41.626521819 (807h34m27.088614302s) [upload=710322]: Next returned nil segment Oct 27 05:05:41.626550404 (807h34m27.088642887s) [upload=710322]: Waiting for error group managing segments... Oct 27 05:05:41.915220559 (807h34m27.377313042s) [noise=true , address=145.79.227.3:4151 , piece_id=5QSRNGVI6YMFA6QDMJEFVQSXOIQGT6ZXG4EMXRDZK66X34LVPKPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.915350577 (807h34m27.37744306s) [noise=true , address=145.79.227.9:4366 , piece_id=KF3FOBVYEVTQ6LFAC6NDFISEPQ5PNS57P3A7P5YJRDSOK6JTDBLA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.921729213 (807h34m27.383821696s) [noise=true , address=145.79.227.5:4222 , piece_id=LA63Z5SP5TVIQMKYOI77PQGOP2FBN5RJGSPRKNSESI3QNVU6EDDA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.928003130 (807h34m27.390095613s) [noise=true , address=145.79.227.13:4804 , piece_id=YPC3ADWSYQKFTAPIRNCACG6GIUY5QW4PGM3ZF2IJXGFMZNSZ5F7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.931584239 (807h34m27.393676722s) [noise=true , address=145.79.227.19:4078 , piece_id=QKC2OPQ6DEO2MWXVFTV7HJ72ED3RUJ27ZOUINUEZSFYRQEM55MYQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.935671622 (807h34m27.397764105s) [noise=true , address=145.79.227.2:4114 , piece_id=3C47AKDNVG6UMFKHO7A7H4K23BWMSNLSQUEK2TS3RHGQCMWFIHHA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.940149509 (807h34m27.402241992s) [noise=true , address=145.79.227.3:4150 , piece_id=QGTF5F7WFARJR7D66ZGKQMYX7ZNWN2R4HEK35S7BZBQVBPQ6CU7Q, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.955862480 (807h34m27.417954963s) [noise=true , address=145.79.227.11:4729 , piece_id=OKLD3FLJNX6SNIHKUWD4HUVY5N7EYZH7ISJLZ63XDV66DK32QWTQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.958842093 (807h34m27.420934566s) [noise=true , address=145.79.227.20:4438 , piece_id=MVFT76SNAZINMZFXBWO4DS2CRURB5FARPUTN4VKQHPX7R3GJTPPQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.967659794 (807h34m27.429752277s) [noise=true , address=145.79.227.14:4839 , piece_id=EAY63EYQCGMQNP4WDHOOLHFZT7CRTQ35BHBEYLPDH6BHPXKOYI7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.987776260 (807h34m27.449868733s) [noise=true , address=145.79.227.18:4005 , piece_id=ME54PWVZCTEERW4PEOO6YOMQ7GZGQL3S44WBFRIWFAU3HOJOOGPA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:41.995802724 (807h34m27.457895207s) [noise=true , address=145.79.227.24:4582 , piece_id=5OFSKA27GOEJQDDKXJ5GUOET5UF7IPFWSX4GOZS2MLZRNXHM4V7A, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020054403 (807h34m27.482146886s) [noise=true , address=145.79.227.26:4655 , piece_id=3CDADXPCUW6UQOM3WJ3LBSSDCOAJWEKWAKCBIFHACAMA6663A4XA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: <nil> Oct 27 05:05:42.020084941 (807h34m27.482177425s) [seg_pos={0 0}, upload=710322]: Segment reached optimal threshold of 13 pieces. Oct 27 05:05:42.020235880 (807h34m27.482328363s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020247902 (807h34m27.482340385s) [noise=true , address=45.91.116.6:4973 , piece_id=SMPDX525OZ6GM4AKLZ6JZ5QMESHUZQT4IWXSMLSLPPBEJJQICCKA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020367269 (807h34m27.482459752s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020374003 (807h34m27.482466476s) [noise=true , address=45.91.116.4:4939 , piece_id=Q3GCAC2QQY3N3N3RJZIXDAXWK43HDB7QOHY6MFUMW5XM5YZQVDQA, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020656470 (807h34m27.482748953s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Oct 27 05:05:42.020664706 (807h34m27.482757179s) [noise=true , address=45.91.116.12:4227 , piece_id=LTVAIEVDMFW6BPYENKBRYHS7LAF2VWO7SCA63AYRQKUB5YO6YREQ, seg_pos={0 0}, upload=710322]: Long tail cancellation due to optimal threshold Oct 27 05:05:42.020682008 (807h34m27.482774481s) [seg_pos={0 0}, upload=710322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 27 05:05:42.020775817 (807h34m27.48286829s) [upload=710322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 27 05:05:42.020781618 (807h34m27.482874091s) [upload=710322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 27 05:05:42.328926888 (807h34m27.791019361s) [upload=710322]: Done uploading segments... Oct 27 05:05:42.328948129 (807h34m27.791040612s) [upload=710322]: Upload finished. err: <nil> Duration: 5.940432057s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)