Orbiter History

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 251.977347ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:32:03 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 252.007235ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 248.466852ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:31:02 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 248.495106ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 262.698833ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:30:31 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 262.72346ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 291.178914ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:29:24 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 291.204783ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 253.810268ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:28:17 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 253.837531ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 255.447214ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:28:00 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 255.573835ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 250.860881ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:26:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 250.884065ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 256.098805ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:25:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.129334ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.42660732s remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:24:39 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 3.427338353s edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 2m0.000956484s remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/387e4bbadec490d1 FAIL (2025-12-24 19:24:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 2m0.000980339s edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.83417518s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:27:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.834199987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.196180451s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:26:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.196210278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.976982841s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:25:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.977007166s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.397817055s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:24:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.397845761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.928359103s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:23:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.928383349s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.838903634s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:23:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.838933269s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.958804738s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:22:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.958841498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.433460343s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:21:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.43349072s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.761310182s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:20:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.761339047s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.051279324s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:19:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.051310403s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.820062729s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:18:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.820086504s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.82106227s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:18:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.821091567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.956843268s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:17:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.956862284s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.044055957s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:16:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.044078709s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.477351662s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:15:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.477377301s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.854186161s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:14:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.854230055s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.056340567s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:13:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.056371477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.980255529s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:13:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.980284714s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.436113107s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:12:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.436137884s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 7.005475596s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:11:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.005501856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.193597938s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:11:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.19362469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.848588003s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:10:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.848619953s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.170978208s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:09:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.170993126s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.785271828s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:08:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.785300613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.270840202s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:07:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.270879827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.865729867s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:07:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.865757148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.904882856s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:06:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.904922122s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.322239574s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:05:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.322267186s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.850462722s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:04:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.850494173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.828210401s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:04:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.828232975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.001309658s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:04:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.001336039s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.966725445s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:03:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.96675498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.746415431s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:02:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.746432302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.9208166s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:02:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.920843079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.387680784s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:01:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.387712193s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.78110825s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 18:00:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.781139149s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.833677785s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 17:59:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.833704365s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.422920177s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 17:59:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.422951235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.998564721s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 17:58:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.998593055s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.013666802s trace: https://jaeger.dev.storj.io/trace/64a70b063120f6ff FAIL (2025-12-24 17:57:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 24 17:41:34.817232055 (458h12m34.68831509s) [upload=402229]: Starting upload Dec 24 17:41:34.817285716 (458h12m34.688368751s) [upload=402229]: Done starting upload Dec 24 17:41:34.817371771 (458h12m34.688454806s) [upload=402229]: Uploading segments... Dec 24 17:41:34.817473855 (458h12m34.68855689s) [upload=402229]: Got next segment. Inline: false Dec 24 17:41:34.817508080 (458h12m34.688591115s) [upload=402229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 24 17:41:34.817512659 (458h12m34.688595694s) [upload=402229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 24 17:41:35.067154038 (458h12m34.938237083s) [seg_pos={0 0}, upload=402229]: Begin upload segment... Dec 24 17:41:35.067209534 (458h12m34.938292579s) [seg_pos={0 0}, upload=402229]: Done begin upload segment. Dec 24 17:41:35.067237456 (458h12m34.938320491s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067270990 (458h12m34.938354025s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.067737579 (458h12m34.938820624s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068069410 (458h12m34.939152455s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068471236 (458h12m34.939554281s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.068826392 (458h12m34.939909437s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069118318 (458h12m34.940201353s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069520844 (458h12m34.94060389s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.069964640 (458h12m34.941047675s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.070342990 (458h12m34.941426035s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071430701 (458h12m34.942513736s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.071491077 (458h12m34.942574112s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072542138 (458h12m34.943625183s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.072669880 (458h12m34.943752925s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.073725251 (458h12m34.944808286s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:35.075362869 (458h12m34.946445904s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Uploading piece... Dec 24 17:41:36.870712973 (458h12m36.741796018s) [upload=402229]: Next returned nil segment Dec 24 17:41:36.870729545 (458h12m36.74181258s) [upload=402229]: Waiting for error group managing segments... Dec 24 17:41:37.160017607 (458h12m37.031100642s) [noise=true , address=145.79.227.32:4834 , piece_id=OI4HEZS66F5LGW33SYDOAD4CDE47KP3MZ3C53BLTJWTV7D6AYK6Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.172949711 (458h12m37.044032757s) [noise=true , address=145.79.227.45:4231 , piece_id=O2RJOW4SBW2XC3PJ5XJ5NLPCTUZXQQBYZDGOBX3YZTW6FDVTOFMA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.183665827 (458h12m37.054748862s) [noise=true , address=145.79.227.50:4374 , piece_id=MA5VLFQAK6U245SGQR5UODBBIQD7CPICYCYTM5SL3GMCLXURWZ7A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.245183227 (458h12m37.116266262s) [noise=true , address=145.79.227.7:4294 , piece_id=RAWBIZM3YPIHZ64SK7KXOXTL4GPVK3JGRGDINJN57USLVX53T6TA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.324356015 (458h12m37.19543905s) [noise=true , address=45.91.116.14:4191 , piece_id=PZSZV2XFKZOJ5W5HP22PWW776L4ATWJZBEEQTJV6JEHYSIKBS35Q, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.442771707 (458h12m37.313854752s) [noise=true , address=45.91.116.11:4117 , piece_id=IMHTEB3WAFAO6554DFSOVOLTOIU65TI6KQSBEJNVP4W3M6STQYDA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.478430989 (458h12m37.349514054s) [noise=true , address=145.79.227.51:4411 , piece_id=3RKCEBKZSY2MSVPVE7CBSF32WX7APWEIGNX6QRZ6KEN3NYGIMBCQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:37.926537889 (458h12m37.797620924s) [noise=true , address=145.79.227.33:4869 , piece_id=VVRKB2PIP4RRW4DZ3OA67MRRVCKPTACQPY633LHHLLZBUNVPKJGQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.187478223 (458h12m38.058561268s) [noise=true , address=145.79.227.52:4446 , piece_id=NJF5KORJX4IZ2VFF3I7GTKELUFDRCZMZK2GZOUNFPURAPUDT47YQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.189416954 (458h12m38.060499999s) [noise=true , address=145.79.227.46:4603 , piece_id=ILEBQVEFNXVBYTGLFBMG4BRO3HNAJ5GYPH53IQF2V7RRSOLHFIQQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.202496650 (458h12m38.073579685s) [noise=true , address=145.79.227.38:4979 , piece_id=3SSLNJ545M7DSFJWO2V7K4AOQMXIXQUAXOAOQIX6NP757T22VWGA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:41:38.237024437 (458h12m38.108107492s) [noise=true , address=145.79.227.4:4187 , piece_id=IAJZIMUQQETQWLVZX6J2UU6RBBHHL5IZNOEBRLV5ZDE6A5MOPEYQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: <nil> Dec 24 17:42:04.130206679 (458h13m4.001289724s) [noise=true , address=145.79.227.49:4339 , piece_id=HBGK2U2V2OIPYUOBXJPV7O7FCP2TB2MAB44WRQSO4R7FYOHV3ROA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37998->145.79.227.49:4339: use of closed network connection Dec 24 17:42:04.130319203 (458h13m4.001402248s) [noise=true , address=145.79.227.11:4730 , piece_id=BNYY6GVOVFYX6YSQR7F23ASYKM5SJWHGK5NQIYTDMO3SGIXU6TNQ, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43852->145.79.227.11:4730: use of closed network connection Dec 24 17:42:04.130411048 (458h13m4.001494083s) [noise=true , address=145.79.227.37:4566 , piece_id=JMK6ZS4UFM3ETIID532YYHTAO2CIYHEJBEU4Z3LLT32J6QL5VY5A, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44142->145.79.227.37:4566: use of closed network connection Dec 24 17:42:04.130495238 (458h13m4.001578283s) [noise=true , address=145.79.227.29:4941 , piece_id=T3QBPFJWPSOCGUJ6C4RC3YGRTTVIDHCUKKL6GYAJRNREPIUM2OCA, seg_pos={0 0}, upload=402229]: Done uploading piece. err: ecclient: upload failed (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni, address:145.79.227.29:4941): protocol: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40326->145.79.227.29:4941: use of closed network connection Dec 24 17:42:04.130522550 (458h13m4.001605595s) [seg_pos={0 0}, upload=402229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 24 17:42:04.370980973 (458h13m4.242064028s) [upload=402229]: Done uploading segments... Dec 24 17:42:04.371031650 (458h13m4.242114695s) [upload=402229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.013693753s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)