Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.013286575s FAIL (2025-10-23 11:38:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.013317974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.937869325s FAIL (2025-10-23 11:38:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 5.937904894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.292676207s FAIL (2025-10-23 11:37:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 8.292713239s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.939064232s FAIL (2025-10-23 11:37:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.939117453s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.070706588s FAIL (2025-10-23 11:37:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 6.070738868s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.187714153s FAIL (2025-10-23 11:36:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 6.18774424s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.962669438s FAIL (2025-10-23 11:36:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.962706409s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.065051084s FAIL (2025-10-23 11:35:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 6.065078346s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.961028105s FAIL (2025-10-23 11:35:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.961132342s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.949679926s FAIL (2025-10-23 11:35:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 5.949764487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.687345539s FAIL (2025-10-23 11:34:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.687380046s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.951567352s FAIL (2025-10-23 11:34:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 5.95160806s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.934480028s FAIL (2025-10-23 11:33:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.934505077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.016706549s FAIL (2025-10-23 11:33:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 6.016740163s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 21.859463017s FAIL (2025-10-23 11:32:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 21.859480289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.891201453s FAIL (2025-10-23 11:32:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.891224727s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.246551696s FAIL (2025-10-23 11:32:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ents... Oct 23 11:31:31.025502072 (718h0m16.487594545s) [upload=631613]: Got next segment. Inline: false Oct 23 11:31:31.025532009 (718h0m16.487624492s) [upload=631613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 11:31:31.025536568 (718h0m16.487629051s) [upload=631613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 11:31:31.293502040 (718h0m16.755594524s) [seg_pos={0 0}, upload=631613]: Begin upload segment... Oct 23 11:31:31.293576993 (718h0m16.755669476s) [seg_pos={0 0}, upload=631613]: Done begin upload segment. Oct 23 11:31:31.293614254 (718h0m16.755706737s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.293645163 (718h0m16.755737646s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294377137 (718h0m16.75646962s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.294744866 (718h0m16.756837339s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295060107 (718h0m16.75715259s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295523449 (718h0m16.757615933s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.295972976 (718h0m16.758065459s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.296650265 (718h0m16.758742748s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297332184 (718h0m16.759424667s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.297776991 (718h0m16.759869464s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.298758580 (718h0m16.760851063s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.299787059 (718h0m16.761879542s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.301378729 (718h0m16.763471212s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.303181892 (718h0m16.765274386s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.304832666 (718h0m16.76692516s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.308342470 (718h0m16.770434953s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.309207536 (718h0m16.771300039s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:31.313177056 (718h0m16.775269549s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Uploading piece... Oct 23 11:31:32.878794349 (718h0m18.340886822s) [upload=631613]: Next returned nil segment Oct 23 11:31:32.878862920 (718h0m18.340955403s) [upload=631613]: Waiting for error group managing segments... Oct 23 11:31:33.283194573 (718h0m18.745287076s) [noise=true , address=77.90.31.11:4588 , piece_id=E33W5T24WAKDF4L3NXCXKGWQLZOOEKQNXFUGQLUIWHE7VQ3HR6CQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.285852094 (718h0m18.747944577s) [noise=true , address=77.90.31.18:4815 , piece_id=E7MWA3IGWCW2HQF4GUEQAEFI2YX5LFAH4245PB3HO2X24PMDZZWQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.316844890 (718h0m18.778937373s) [noise=true , address=45.91.116.7:4040 , piece_id=U3V4FFZGFWCMOGFT6ACAFTKZODOB5YEUYHKX2OWEFJSXXXP7D7UQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.356409389 (718h0m18.818501872s) [noise=true , address=45.91.116.12:4239 , piece_id=6B4XU3WRYHQ3GX2QV5K4Q6EWG6LDGCWPBSKXY3BC7TA643CX55EA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.358993130 (718h0m18.821085613s) [noise=true , address=77.90.31.19:4868 , piece_id=TVWXIIYN7SLHY4EMSG3ZXRMJLQ4EY536FPNP6T26CNRJ45OS6Z5Q, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.384622624 (718h0m18.846715107s) [noise=true , address=77.90.31.10:4529 , piece_id=HNWS6IDOXLH2J2LIVEEJWUIKY4HVHHFJ5KNQP4IA2WLMN5WMX4TA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.393367376 (718h0m18.855459859s) [noise=true , address=77.90.31.16:4753 , piece_id=TTJNKLCSAN32AV3MOAVVFQCZ3M24WTWWD4TCGHF2QJK5ZJ2AL6UA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.416895519 (718h0m18.878988002s) [noise=true , address=45.91.116.5:4315 , piece_id=2PZSVYUTILRUZVG5ZRNJQPJQITKS4PJUXJLETIII66YRCP7KBQLQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.425298731 (718h0m18.887391214s) [noise=true , address=77.90.31.17:4799 , piece_id=OGDGC7DAPTFCSXG53MTLPBIWLTFSYAMFICAELV7DXPZWQ7JNMOQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.458527425 (718h0m18.920619898s) [noise=true , address=45.91.116.11:4143 , piece_id=LECTGB4DNSNTIPQ7HF7GLOG4JMW7GULJNBX5MKTEKCE6D32XHPQA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.496510443 (718h0m18.958602916s) [noise=true , address=45.91.116.9:4283 , piece_id=Y3F7FJKAW4UIPKMXIAFIOIYDX4BS3X3LGLDHODL72D6T6B7DDO4A, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.529020308 (718h0m18.991112781s) [noise=true , address=77.90.31.15:4729 , piece_id=I7LYRJ67NFLNYE6PMDABV2M7CSK2UJ3MD5NYCQGH73JDP27TSUUA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:31:33.593192024 (718h0m19.055284507s) [noise=true , address=77.90.31.21:4889 , piece_id=UWNFS4JJ4OCTOZPISKOJWTPTDII4AGV6GXOLDGYUEFZXY46HZUPA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: <nil> Oct 23 11:32:30.330619309 (718h1m15.792711792s) [noise=true , address=145.79.227.50:4401 , piece_id=EFKR4NN6SJDXQFZNLGCTZGIHDKU6P7JLZ63SHTJTWLLOVKXCPAAQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12p25affFUZgCByWE2j1FS5k7tedvZsHeCcmwMeerMXPFNf8eEY, address:145.79.227.50:4401): protocol: read tcp 188.166.225.170:32946->145.79.227.50:4401: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330696065 (718h1m15.792788538s) [noise=true , address=145.79.227.47:4285 , piece_id=UVJFA7REUZLWGZYEYHXB4CFK6UGXEL5VVGWRZOBI44Q5GTENWGHA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:145.79.227.47:4285): protocol: read tcp 188.166.225.170:54628->145.79.227.47:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330771259 (718h1m15.792863742s) [noise=true , address=145.79.227.40:4062 , piece_id=4ACVXAWBV72FO7XKHNUCBKCZ5GGGYZ3H3GO4LIEFEKABJIMWMRBA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:30.330817136 (718h1m15.792909609s) [noise=true , address=145.79.227.29:4947 , piece_id=UUHAP7U5ZYCVRDTYQRS7RANJ7HBWWRTKDD4LFNQPPAXZH6EJGISA, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU, address:145.79.227.29:4947): protocol: context deadline exceeded; context deadline exceeded Oct 23 11:32:30.330934990 (718h1m15.793027473s) [noise=true , address=145.79.227.31:4822 , piece_id=Z7A5GY7I5JTECZSSNKGM6K6LKZH7ABTS5ONYWO4ZVF6TGMZNMROQ, seg_pos={0 0}, upload=631613]: Done uploading piece. err: ecclient: upload failed (node:127uUPZhKGPaiMVxTEH919M1WvLqDM9VLgaBkuaTBuPbdaXKLLG, address:145.79.227.31:4822): protocol: read tcp 188.166.225.170:46850->145.79.227.31:4822: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 23 11:32:30.330967091 (718h1m15.793059564s) [seg_pos={0 0}, upload=631613]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 23 11:32:30.575020441 (718h1m16.037112924s) [upload=631613]: Done uploading segments... Oct 23 11:32:30.575050358 (718h1m16.037142851s) [upload=631613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.246580521s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.244044671s FAIL (2025-10-23 11:32:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690456804 (718h0m34.152549287s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.690872646 (718h0m34.152965129s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691285382 (718h0m34.153377855s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.691668792 (718h0m34.153761275s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692051451 (718h0m34.154143934s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.692519563 (718h0m34.154612046s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693109987 (718h0m34.15520247s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.693627412 (718h0m34.155719895s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695208062 (718h0m34.157300535s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.695245022 (718h0m34.157337495s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.696963003 (718h0m34.159055486s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.699114781 (718h0m34.161207274s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:48.701035708 (718h0m34.163128191s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Uploading piece... Oct 23 11:31:50.481046142 (718h0m35.943138625s) [upload=631616]: Next returned nil segment Oct 23 11:31:50.481069187 (718h0m35.94316168s) [upload=631616]: Waiting for error group managing segments... Oct 23 11:31:50.827047413 (718h0m36.289139896s) [noise=true , address=45.91.116.13:4154 , piece_id=K4LAX4ARUTWZVUIFJYWYKNYQY3JGJ2SKU6FNZBYCCAYXSYGG4JVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.871438262 (718h0m36.333530755s) [noise=true , address=77.90.31.15:4699 , piece_id=4FYFKFWOAHLCOOS2JKCOV4PRCUAZZ7RMHQ7ZZJGOHA64UT2Y43BA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:31:50.927316536 (718h0m36.389409009s) [noise=true , address=77.90.31.12:4592 , piece_id=XWTZE5IDTGZQWT76K2EP6UBILYPU3CZ6SUNSYAMAZPJ26A4I6HYA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: <nil> Oct 23 11:32:17.743342216 (718h1m3.205434689s) [noise=true , address=145.79.227.16:4909 , piece_id=ECDDS6LEWRO5PP5LBHTX7DVYWH2VJIVNLZ2VIJLTJR4E4R6OZ27A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743421767 (718h1m3.20551426s) [noise=true , address=145.79.227.3:4149 , piece_id=S7YYQ6LAMGGMHDD72FGKSZPHB7RQ32OQGBAK2USILZTA4K7AYTRA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743578346 (718h1m3.205670829s) [noise=true , address=145.79.227.18:4005 , piece_id=EXGEPKKFW437NQKWSVDGLYLXKYDLSXJUX73IADWRMUN4EXFVENVA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743666102 (718h1m3.205758595s) [noise=true , address=145.79.227.8:4329 , piece_id=BGYJYJ3DQHNTBEO5VGB7DDX7O5ORMMV2QEQWGW6N2YVDAH2U7ANQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743773477 (718h1m3.20586596s) [noise=true , address=145.79.227.19:4078 , piece_id=WK4WKEYW3M5JN2WABQDKI4AWI7HVOLCB57OXZINVC3D7VT6XOVHQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743818393 (718h1m3.205910876s) [noise=true , address=145.79.227.12:4765 , piece_id=FIAFAU6I54WOBESSXR74YPBVVGB4JCAOV3V3JGOXUF5TI2LP5VYQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743949432 (718h1m3.206041915s) [noise=true , address=145.79.227.9:4366 , piece_id=ITJARDLREX2M3O2Y56FHXL7CN4VMJVGLC52TWZFPFU27LBJ6RIXA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: rpc: dial tcp 145.79.227.9:4366: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.743967877 (718h1m3.20606036s) [noise=true , address=145.79.227.22:4509 , piece_id=FZZLTKVAGZHVVAFNEX3VVARZINI6BBKLSULSL5UV6SFWGQENCLSQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1229E58YirF37sZU6hfZGrDNFSxQDYsWSo4x3mG9BoYsNsLNtjy, address:145.79.227.22:4509): protocol: read tcp 188.166.225.170:54808->145.79.227.22:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744058890 (718h1m3.206151373s) [noise=true , address=145.79.227.15:4873 , piece_id=G236QF5XILOYRRS6XZGEZLZZRCYPQA5PF5OAMMDPH7EG52LKGEOA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57766->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744152779 (718h1m3.206245262s) [noise=true , address=145.79.227.5:4221 , piece_id=3MLV7IAQBHC4TNTO6Z6MU63EJULUAIIIFRSZIKEDNOEIBYQWRG2A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:58548->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744170483 (718h1m3.206262966s) [noise=true , address=145.79.227.10:4402 , piece_id=BVKFMYUIYQGUBSBDLSAYWHZ7HTRPYRRWXLFDA63WDIUBDKPX3C3A, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: read tcp 188.166.225.170:53316->145.79.227.10:4402: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744261245 (718h1m3.206353719s) [noise=true , address=145.79.227.13:4803 , piece_id=OFIK5J2DJ4FYSDXETP5LN2HN5D2DVZSZODPHEHD5YRBC4OCOS6PA, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60182->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744294208 (718h1m3.206386691s) [noise=true , address=145.79.227.14:4839 , piece_id=VZSETJNSU54VHHXSGF6EJ3QP76WYTNEKY4GSXQSKIW6OFHUNYNKQ, seg_pos={0 0}, upload=631616]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: read tcp 188.166.225.170:34924->145.79.227.14:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 23 11:32:17.744321941 (718h1m3.206414424s) [seg_pos={0 0}, upload=631616]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Oct 23 11:32:17.985568445 (718h1m3.447660938s) [upload=631616]: Done uploading segments... Oct 23 11:32:17.985602079 (718h1m3.447694562s) [upload=631616]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.244073486s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 13.589888681s FAIL (2025-10-23 11:00:55 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 13.589920482s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 3.887988131s FAIL (2025-10-23 11:00:50 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 3.888022797s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 3.531065002s FAIL (2025-10-23 11:00:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 3.531102603s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.168432042s FAIL (2025-10-23 10:59:56 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.168465966s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.073293002s FAIL (2025-10-23 10:59:40 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.073383213s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.400328801s FAIL (2025-10-23 10:58:42 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.400365048s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.115907549s FAIL (2025-10-23 10:58:37 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.115961491s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.370069209s FAIL (2025-10-23 10:58:25 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.370099265s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.069654284s FAIL (2025-10-23 10:58:11 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.069691746s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.282127443s FAIL (2025-10-23 10:57:48 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.28215722s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.143751289s FAIL (2025-10-23 10:57:06 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.143784591s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.098330912s FAIL (2025-10-23 10:57:05 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.098359686s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.365971446s FAIL (2025-10-23 10:56:27 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.366007746s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.089616995s FAIL (2025-10-23 10:55:55 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.08965086s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.054071527s FAIL (2025-10-23 10:55:14 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.054107536s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.029223847s FAIL (2025-10-23 10:54:52 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.029260375s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.063315755s FAIL (2025-10-23 10:54:36 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.063360761s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.05977922s FAIL (2025-10-23 10:54:19 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.059809668s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.238134037s FAIL (2025-10-23 10:54:06 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.238158934s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.163995903s FAIL (2025-10-23 10:53:40 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.164029616s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.049291299s FAIL (2025-10-23 10:53:12 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.049324762s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.022311029s FAIL (2025-10-23 10:53:11 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.022347649s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.420076467s FAIL (2025-10-23 10:52:38 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.420110451s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.480366357s FAIL (2025-10-23 10:52:19 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.480398097s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.04293768s FAIL (2025-10-23 10:52:09 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.04297468s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.095389159s FAIL (2025-10-23 10:51:36 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.095424354s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.944675171s FAIL (2025-10-23 10:51:35 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 5.944705218s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.062673257s FAIL (2025-10-23 10:50:31 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.062713084s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.000821162s FAIL (2025-10-23 10:50:22 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.000854365s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.149705702s FAIL (2025-10-23 10:49:39 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.149775195s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.033486862s FAIL (2025-10-23 10:49:38 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.033516679s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.068089027s FAIL (2025-10-23 10:48:56 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 42) Duration: 6.068124184s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)