Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.127899923s FAIL (2025-09-08 09:04:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.127928989s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.387301765s FAIL (2025-09-08 09:03:37 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.387328357s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.354955508s FAIL (2025-09-08 09:02:34 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.354977952s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.271639948s FAIL (2025-09-08 09:02:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.271704742s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.538634455s FAIL (2025-09-08 09:01:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.538665776s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.177116974s FAIL (2025-09-08 09:00:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.17714649s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.776795264s FAIL (2025-09-08 09:00:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.776835361s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.951830943s FAIL (2025-09-08 08:59:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.951861601s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.391591384s FAIL (2025-09-08 08:58:49 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.391638815s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 4.980579154s FAIL (2025-09-08 08:58:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.980605255s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.277854265s FAIL (2025-09-08 08:57:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.277890284s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.263106638s FAIL (2025-09-08 08:56:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.263135303s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.289905978s FAIL (2025-09-08 08:55:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.289936867s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.060073378s FAIL (2025-09-08 08:54:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.060102083s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.367105449s FAIL (2025-09-08 08:53:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.367148299s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.10322617s FAIL (2025-09-08 08:52:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.103255886s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.355769919s FAIL (2025-09-08 08:52:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.355789978s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m50.982119097s FAIL (2025-09-08 08:52:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1m50.982144105s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.00201704s FAIL (2025-09-08 01:51:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.002047356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.025881602s FAIL (2025-09-08 01:51:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.025912541s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.890406244s FAIL (2025-09-08 01:50:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.890431112s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.934219647s FAIL (2025-09-08 01:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.934251367s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.884851534s FAIL (2025-09-08 01:48:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.88488026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.017728907s FAIL (2025-09-08 01:47:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.017762862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.888554723s FAIL (2025-09-08 01:47:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.888583378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.08276183s FAIL (2025-09-08 01:46:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.082784815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.885098315s FAIL (2025-09-08 01:46:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.885130878s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.912823936s FAIL (2025-09-08 01:45:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.912851548s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.061560895s FAIL (2025-09-08 01:44:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.061587525s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.812411251s FAIL (2025-09-08 01:44:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.81244138s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.129561545s FAIL (2025-09-08 01:44:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.129582976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.089241019s FAIL (2025-09-08 01:44:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.089286817s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.366452042s FAIL (2025-09-08 01:43:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.366475076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.120819554s FAIL (2025-09-08 01:43:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.120848138s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.038397075s FAIL (2025-09-08 01:42:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.03842662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.154700716s FAIL (2025-09-08 01:42:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.15473471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.913000305s FAIL (2025-09-08 01:41:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.913023959s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.133395062s FAIL (2025-09-08 01:41:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.133445459s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.075692001s FAIL (2025-09-08 01:41:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.075737727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.13484399s FAIL (2025-09-08 01:41:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.134893384s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.281857713s FAIL (2025-09-08 01:40:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.281888401s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.233859062s FAIL (2025-09-08 01:40:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.233887236s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.228254737s FAIL (2025-09-08 01:39:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.228284555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.068701668s FAIL (2025-09-08 01:39:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.068734821s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.046889274s FAIL (2025-09-08 01:38:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.046928679s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.978983623s FAIL (2025-09-08 01:38:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.97901258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.045994635s FAIL (2025-09-08 01:37:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.046021757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.952205092s FAIL (2025-09-08 01:37:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.952223106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.169528194s FAIL (2025-09-08 01:37:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.169558903s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.95849513s FAIL (2025-09-08 01:36:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.958525088s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)