Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 30.001000127s FAIL (2025-09-06 12:26:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:25:18.562165527 (190h17m15.242361096s) [upload=167422]: Starting upload Sep 6 12:25:18.562223879 (190h17m15.242419448s) [upload=167422]: Done starting upload Sep 6 12:25:18.562274195 (190h17m15.242469764s) [upload=167422]: Uploading segments... Sep 6 12:25:18.562308862 (190h17m15.242504421s) [upload=167422]: Got next segment. Inline: true Sep 6 12:25:18.562312749 (190h17m15.242508318s) [upload=167422]: Waiting for error group managing segments... Sep 6 12:25:18.562344029 (190h17m15.242539598s) [upload=167422]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:25:18.562347926 (190h17m15.242543495s) [upload=167422]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 12:25:18.562350531 (190h17m15.2425461s) [upload=167422]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:25:31.870144262 (190h17m28.550339831s) [upload=167422]: Done uploading segments... Sep 6 12:25:31.870180752 (190h17m28.550376321s) [upload=167422]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001020906s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.000800627s FAIL (2025-09-06 12:26:31 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.000820144s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 26.706163171s FAIL (2025-09-06 12:26:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/uplink-tests/from_do_prod_ops_orbiter_ap_0/1k):: after a3853fb5adf9bc2d466defe7ba62d026318d37e5af3b6294bb8336fc60c5232f != before 37af7165d94ac401618488529c9bea822a33da5756118671f1e59c67c41ee22c ======LOGS====== Sep 6 12:26:13.021897432 (190h18m9.702093001s) [upload=167435]: Starting upload Sep 6 12:26:13.022003435 (190h18m9.702199004s) [upload=167435]: Done starting upload Sep 6 12:26:13.022095141 (190h18m9.70229071s) [upload=167435]: Uploading segments... Sep 6 12:26:13.022178731 (190h18m9.7023743s) [upload=167435]: Got next segment. Inline: true Sep 6 12:26:13.022183480 (190h18m9.702379049s) [upload=167435]: Waiting for error group managing segments... Sep 6 12:26:13.022226332 (190h18m9.702421901s) [upload=167435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:26:13.022230971 (190h18m9.70242654s) [upload=167435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 12:26:13.022233807 (190h18m9.702429376s) [upload=167435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:26:24.097913192 (190h18m20.778108821s) [upload=167435]: Done uploading segments... Sep 6 12:26:24.097935716 (190h18m20.778131345s) [upload=167435]: Upload finished. err: <nil> Duration:* 26.706185705s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 13.449194475s FAIL (2025-09-06 12:26:20 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 13.449212039s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............X......X......................X......X.....X.XXX] duration: 31.379918152s FAIL (2025-09-06 12:26:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== Sep 6 12:25:52.313956579 (190h17m48.994152138s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3JQXBHXA3QE35YWFJWQFF4RGOMBV5HZR33CQNPFMV5MVKKCL4UA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314195115 (190h17m48.994390694s) [noise=true , address=45.91.116.8:4045 , piece_id=QR4BHHVXFPBGZNWQUVZRMZQX7NFOA2NVYIZAUXGCXSXO7CFJJBZQ, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314236033 (190h17m48.994431602s) [noise=true , address=45.91.116.5:4298 , piece_id=QTCD5USILX7DHD2LXDTOEGXA7GCXGMZZFJ276U5N3MEHBA67AFKA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314511350 (190h17m48.994706909s) [noise=true , address=77.90.31.17:4770 , piece_id=FDO5IKXHHPUHFB5X27DZ3KTYO743DAJRM2G36MBLBFPPEY2EVSBA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314533703 (190h17m48.994729272s) [noise=true , address=45.91.116.7:4010 , piece_id=YJICRXNO6BXPUASDJ73UXU7FIVATEGZPEJSTTM4YLGABE5A66RCQ, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314751029 (190h17m48.994946598s) [noise=true , address=45.91.116.14:4191 , piece_id=E42IJAE2WHH2GHCVPEJQYUY73CZI3CYD5PK3P4IJCKFHPJ533D4A, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314759254 (190h17m48.994954823s) [noise=true , address=45.91.116.10:4082 , piece_id=5E7J43YJDL73BUB7RQROZ6CIE5WEEEYCVAFZOKOFLZGRUAVLRHJA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.314986419 (190h17m48.995181979s) [noise=true , address=45.91.116.14:4189 , piece_id=O7HZWOF2G7GUALYIWGGP3IFPP7MPD2ULQ26XBTMEOVV6KKA57EWA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.315106980 (190h17m48.995302549s) [noise=true , address=45.91.116.7:4011 , piece_id=EZRC4XYPQO7SRDYFJOS7SNCTQOFOBOGQ26BWXNKUGDDJO3CJ7RTA, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:25:52.315258660 (190h17m48.99545423s) [noise=true , address=45.91.116.8:4047 , piece_id=6DUFOLLAZ3BWWIHRZYOEUXWFND7KDHOJRL67YBPGHGTZPAA5DXYQ, seg_pos={0 0}, upload=167428]: Uploading piece... Sep 6 12:26:15.462826440 (190h18m12.143022069s) [upload=167428]: Next segment err: context deadline exceeded Sep 6 12:26:15.463825332 (190h18m12.144020921s) [noise=true , address=45.91.116.6:4975 , piece_id=TLFEDZXQSQJRYVD6UHVGUUOGYZEBSUEY72WVUS7P3T66223EZLAA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 188.166.225.170:46858->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.463909032 (190h18m12.144104641s) [noise=true , address=45.91.116.7:4010 , piece_id=YJICRXNO6BXPUASDJ73UXU7FIVATEGZPEJSTTM4YLGABE5A66RCQ, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: read tcp 188.166.225.170:55574->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.463927918 (190h18m12.144123497s) [noise=true , address=45.91.116.7:4011 , piece_id=EZRC4XYPQO7SRDYFJOS7SNCTQOFOBOGQ26BWXNKUGDDJO3CJ7RTA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: rpc: dial tcp 45.91.116.7:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.463987312 (190h18m12.144182931s) [noise=true , address=45.91.116.14:4189 , piece_id=O7HZWOF2G7GUALYIWGGP3IFPP7MPD2ULQ26XBTMEOVV6KKA57EWA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 188.166.225.170:60818->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464043369 (190h18m12.144238948s) [noise=true , address=45.91.116.8:4047 , piece_id=6DUFOLLAZ3BWWIHRZYOEUXWFND7KDHOJRL67YBPGHGTZPAA5DXYQ, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: read tcp 188.166.225.170:42022->45.91.116.8:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464088104 (190h18m12.144283683s) [noise=true , address=45.91.116.12:4226 , piece_id=H6N64V7JNHG3SN3AAKPWAJSEORD556T36JLEEGQIN7CUZ5VEXT4Q, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 188.166.225.170:52744->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464173257 (190h18m12.144368836s) [noise=true , address=45.91.116.8:4045 , piece_id=QR4BHHVXFPBGZNWQUVZRMZQX7NFOA2NVYIZAUXGCXSXO7CFJJBZQ, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 188.166.225.170:57760->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464224806 (190h18m12.144420385s) [noise=true , address=45.91.116.5:4298 , piece_id=QTCD5USILX7DHD2LXDTOEGXA7GCXGMZZFJ276U5N3MEHBA67AFKA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 188.166.225.170:39028->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464271225 (190h18m12.144466804s) [noise=true , address=45.91.116.10:4082 , piece_id=5E7J43YJDL73BUB7RQROZ6CIE5WEEEYCVAFZOKOFLZGRUAVLRHJA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:45212->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464313947 (190h18m12.144509526s) [noise=true , address=45.91.116.4:4938 , piece_id=WMNLZBFGVBPEJFIBRACYRUWGLYTZMJDFS436VFSFRMWRD7WTNJEA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:42906->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464358492 (190h18m12.144554071s) [noise=true , address=45.91.116.13:4153 , piece_id=LA3NFZMP3NUKJ6CVLTMQLEXZYCN6WOYZ5YYBB24GMXGYPIB6VX3A, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: read tcp 188.166.225.170:60704->45.91.116.13:4153: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464397808 (190h18m12.144593387s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3JQXBHXA3QE35YWFJWQFF4RGOMBV5HZR33CQNPFMV5MVKKCL4UA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 188.166.225.170:52676->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464451430 (190h18m12.144646999s) [noise=true , address=45.91.116.8:4048 , piece_id=YNU2HGAMQE52A5E34UNF5CK4P42JB6ZWLA6ZKYAN5HIAS22UDS7Q, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:59910->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464493150 (190h18m12.144688719s) [noise=true , address=77.90.31.17:4770 , piece_id=FDO5IKXHHPUHFB5X27DZ3KTYO743DAJRM2G36MBLBFPPEY2EVSBA, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:46728->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464519029 (190h18m12.144714598s) [noise=true , address=77.90.31.17:4771 , piece_id=NOBEKPWHJNBZONX46XI75MSGHGDRJHEAL3J6CE4A2KTWNW6OW52A, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: read tcp 188.166.225.170:36796->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464534950 (190h18m12.144730519s) [noise=true , address=45.91.116.14:4191 , piece_id=E42IJAE2WHH2GHCVPEJQYUY73CZI3CYD5PK3P4IJCKFHPJ533D4A, seg_pos={0 0}, upload=167428]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 188.166.225.170:38098->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:15.464565038 (190h18m12.144760617s) [seg_pos={0 0}, upload=167428]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 31.379931618s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.188233472s FAIL (2025-09-06 12:26:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 5) ======LOGS====== write tcp 188.166.225.170:38086->185.167.204.205:4031: use of closed network connection Sep 6 12:26:14.221676059 (190h18m10.901871628s) [noise=true , address=185.167.204.205:4036 , piece_id=WXZW5ET4M7G7SDGOOIVFVPYUM7VCI2JZIWEXQW4WBJAH775EPOLA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12Mngc2aRWoo1ZQGktFfA4DauhKyEFcbRk6dg7nYa1jZUg2srMV, address:185.167.204.205:4036): protocol: read tcp 188.166.225.170:56738->185.167.204.205:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.221774917 (190h18m10.901970486s) [noise=true , address=185.167.204.206:4048 , piece_id=3UXXCBVR425NFQCTTK3Q5AWZFV4GWRIL4SERVX25JLYO3QRUHANA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: read tcp 188.166.225.170:37448->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.221858317 (190h18m10.902053896s) [noise=true , address=185.167.204.206:4068 , piece_id=C2NHYMFQ62WQCEAXJKFLRWKNHD345PXBWLFR5RXUGPCCDJSLPJ2Q, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: read tcp 188.166.225.170:60196->185.167.204.206:4068: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.221895909 (190h18m10.902091479s) [noise=true , address=81.171.14.70:4036 , piece_id=7P7PW2W3G57C5NWTHQLKF4DWAMCL4MB2ITWBDHSOTN5AZR5KWGUQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:38328->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.221926959 (190h18m10.902122528s) [noise=true , address=185.167.204.205:4035 , piece_id=QAGOZCU3MLEPBQD52HXPVWDKEHBCBWCJMOOIZPJQQCY2IAW5WRSA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1TT3Cyz1GPrGa9P9jWePFarF4LFaJfyfRbGM1hH5ctGx65JUJp, address:185.167.204.205:4035): protocol: read tcp 188.166.225.170:39154->185.167.204.205:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.221962697 (190h18m10.902158266s) [noise=true , address=185.167.204.205:4014 , piece_id=LHY3IZ77WBHRR2EJPZ3IXQK6RA2SW3JDCM4SMA3SUXTUUD2XNO3A, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 188.166.225.170:37482->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222001391 (190h18m10.90219696s) [noise=true , address=185.167.204.205:4025 , piece_id=C37O4HNGWAAPFVK5TIIEQO5DG2HPJBAJDFSNQFQ5VSU5STPZZ3WA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: read tcp 188.166.225.170:48770->185.167.204.205:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222032411 (190h18m10.90222797s) [noise=true , address=185.167.204.205:4019 , piece_id=YNTH3IC6MH4IHVKHETCU7ARZ6VHLWLRHHIYZMK3JFTHMWEDILYUQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12CwWyCCjbNNuuSxpk9PhiWxG2yvuT19gDyJhezMfUEez6LrpDj, address:185.167.204.205:4019): protocol: read tcp 188.166.225.170:53224->185.167.204.205:4019: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222061276 (190h18m10.902256845s) [noise=true , address=185.167.204.206:4052 , piece_id=N47I3BW3XLZQVAESBEUYG47DD2W5MHWKXL44BYEWDZ3ODN3E7NEQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12Xr3UmaAcshmxZMecu5i7VHA6LJxiZm9u6wngapEma3hz3tNmn, address:185.167.204.206:4052): protocol: read tcp 188.166.225.170:60594->185.167.204.206:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222066676 (190h18m10.902262235s) [noise=true , address=185.167.204.205:4007 , piece_id=SZMYH73TWYQBX5NKFTWFCM46SEXQCSWVJUYWK2KTGUKWKSY2RWXQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222130999 (190h18m10.902326568s) [noise=true , address=81.171.14.70:4048 , piece_id=VIH37BGS34IS5MR5NZRLYGD3MB4SMXNSV4SZA2SRZMFWKY46HG4A, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: read tcp 188.166.225.170:57988->81.171.14.70:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222202987 (190h18m10.902398546s) [noise=true , address=185.167.204.205:4029 , piece_id=G5KJCARB4QDPHYASJIVKMXAK4YX6E2KXTMQTDZBBH6Y67Y675LFA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: read tcp 188.166.225.170:54494->185.167.204.205:4029: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222227344 (190h18m10.902422903s) [noise=true , address=81.171.14.70:4026 , piece_id=VBFO6TES773SIP4FCFQD326Y44QZBHJ5KBEEQMDUB2ONE237YZFQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56756->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56756->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56756->81.171.14.70:4026: use of closed network connection Sep 6 12:26:14.222252101 (190h18m10.90244767s) [noise=true , address=185.167.204.205:4034 , piece_id=7D6VZZDA3WBLDFOUZYU7AR5GX6NJO4A6CE5D6VBPPIPLH3PA2RFA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: read tcp 188.166.225.170:36576->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222278041 (190h18m10.90247361s) [noise=true , address=185.167.204.206:4037 , piece_id=F57CLFT5KGBBH2PEKHABQMLZYB5MS2KIEPCVZNLNHND3YYDXWNNQ, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12nEq1PMWr39K5LBUcViEtEb1KTnsJLjMaBbNGuqpc13WdZb18R, address:185.167.204.206:4037): protocol: read tcp 188.166.225.170:43956->185.167.204.206:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222291396 (190h18m10.902486965s) [noise=true , address=185.167.204.205:4003 , piece_id=QIMZUXVAMIAUWJKHGBXY56FDV24YGKWDLESMFZJ5Y4F3XOX7T2JA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:1CjWvvJ4FdZc41sRgsQFWz4Y1ATZKgzsvDcUTmBSrCC15ekCmF, address:185.167.204.205:4003): protocol: read tcp 188.166.225.170:57892->185.167.204.205:4003: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222318778 (190h18m10.902514347s) [noise=true , address=185.167.204.206:4055 , piece_id=SWJLK2RDVSGMAQLIFI4HJIO2I2FM3KW4L7BMFGJIJCUEDFOFRIIA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: read tcp 188.166.225.170:51162->185.167.204.206:4055: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222333056 (190h18m10.902528615s) [noise=true , address=185.167.204.206:4062 , piece_id=PBY6KMYJA7HO2G466XJRRAD56WGC63X3Z55NTH3FNYDYXN6ANTKA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 188.166.225.170:46868->185.167.204.206:4062: use of closed network connection; write tcp 188.166.225.170:46868->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46868->185.167.204.206:4062: use of closed network connection Sep 6 12:26:14.222347764 (190h18m10.902543333s) [noise=true , address=185.167.204.206:4072 , piece_id=H3C5UTWKW53UGCRDTD6EGTPMBNI54Y7EIZ4VD2VVELZ6XLJYTZUA, seg_pos={0 0}, upload=167421]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: read tcp 188.166.225.170:59094->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:26:14.222372210 (190h18m10.902567779s) [seg_pos={0 0}, upload=167421]: Done waiting for segment. successful: 5 optimal: 26 errs: <nil> Sep 6 12:26:14.406842811 (190h18m11.08703839s) [upload=167421]: Done uploading segments... Sep 6 12:26:14.406869091 (190h18m11.08706465s) [upload=167421]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 5) Duration: 1m0.188260533s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 10.355120293s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 12:26:12 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 10.355136124s edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 21.383109241s FAIL (2025-09-06 12:26:12 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 21.383137195s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000232139s FAIL (2025-09-06 12:26:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000256747s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.108977688s FAIL (2025-09-06 12:26:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 6.109005983s edge authservice http us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000484482s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 12:26:02 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 30.000514668s edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.000933247s FAIL (2025-09-06 12:26:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 6 12:25:45.985970260 (190h17m42.666165819s) [upload=167425]: Starting upload Sep 6 12:25:45.986037990 (190h17m42.666233559s) [upload=167425]: Done starting upload Sep 6 12:25:45.986125938 (190h17m42.666321507s) [upload=167425]: Uploading segments... Sep 6 12:25:45.986206091 (190h17m42.66640166s) [upload=167425]: Got next segment. Inline: false Sep 6 12:25:45.986238092 (190h17m42.666433661s) [upload=167425]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:25:45.986243332 (190h17m42.666438901s) [upload=167425]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:26:00.418692876 (190h17m57.098888445s) [upload=167425]: Next segment err: context deadline exceeded Sep 6 12:26:00.418727453 (190h17m57.098923022s) [upload=167425]: Done uploading segments... Sep 6 12:26:00.418746319 (190h17m57.098941888s) [upload=167425]: Upload finished. err: context deadline exceeded Duration: 1m0.000945381s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001302973s FAIL (2025-09-06 12:25:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:25:49.060284952 (190h17m45.740480521s) [upload=167427]: Starting upload Sep 6 12:25:49.060340278 (190h17m45.740535847s) [upload=167427]: Done starting upload Sep 6 12:25:49.060413648 (190h17m45.740609217s) [upload=167427]: Uploading segments... Sep 6 12:25:49.060518468 (190h17m45.740714127s) [upload=167427]: Got next segment. Inline: true Sep 6 12:25:49.060528648 (190h17m45.740724257s) [upload=167427]: Waiting for error group managing segments... Sep 6 12:25:49.060584435 (190h17m45.740780004s) [upload=167427]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:25:49.060589064 (190h17m45.740784633s) [upload=167427]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 12:25:49.060591518 (190h17m45.740787087s) [upload=167427]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:25:54.569723814 (190h17m51.249919383s) [upload=167427]: Done uploading segments... Sep 6 12:25:54.569821470 (190h17m51.250017039s) [upload=167427]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.001313613s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000421766s FAIL (2025-09-06 12:25:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000441903s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............X......X......................X......X.....X.XX] duration: 30.000902274s FAIL (2025-09-06 12:25:45 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 30.000924547s core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000520743s FAIL (2025-09-06 12:25:36 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.000550831s edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001270549s FAIL (2025-09-06 12:25:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 12:25:18.562165527 (190h17m15.242361096s) [upload=167422]: Starting upload Sep 6 12:25:18.562223879 (190h17m15.242419448s) [upload=167422]: Done starting upload Sep 6 12:25:18.562274195 (190h17m15.242469764s) [upload=167422]: Uploading segments... Sep 6 12:25:18.562308862 (190h17m15.242504421s) [upload=167422]: Got next segment. Inline: true Sep 6 12:25:18.562312749 (190h17m15.242508318s) [upload=167422]: Waiting for error group managing segments... Sep 6 12:25:18.562344029 (190h17m15.242539598s) [upload=167422]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:25:18.562347926 (190h17m15.242543495s) [upload=167422]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 12:25:18.562350531 (190h17m15.2425461s) [upload=167422]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:25:31.870144262 (190h17m28.550339831s) [upload=167422]: Done uploading segments... Sep 6 12:25:31.870180752 (190h17m28.550376321s) [upload=167422]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001295487s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000512457s FAIL (2025-09-06 12:25:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000531033s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000836568s FAIL (2025-09-06 12:25:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000850114s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000488179s FAIL (2025-09-06 12:25:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.000501886s core uplink us1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001384032s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 12:25:22 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 30.001417356s edge linksharing us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.175571135s FAIL (2025-09-06 12:25:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 11) ======LOGS====== piece_id=OZJVL5UD6X3BBFSHDVGK4ILGPI3DWE7A7V3UWNTVCYJZX6J3HGOQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:15.847979329 (190h16m12.528174898s) [noise=true , address=89.149.201.147:4024 , piece_id=SCMCCWR3Z2WWBSJCVZRDB437H6ZH43YAE7PIVWN254E7ZN3WL7BA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:15.863500689 (190h16m12.543696258s) [noise=true , address=89.149.201.147:4019 , piece_id=UTXFG7N4HMDML3B4AA42NAOVVSFOA5NZBRMBBVA6YZLTWXSHOHLQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:15.879519792 (190h16m12.559715361s) [noise=true , address=89.149.201.147:4013 , piece_id=RLLTHG7Q2EW2Y5I3BYUFRUW3OWZPBP2CMUV3VPF45S7YRGEBGVQA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:15.902229489 (190h16m12.582425058s) [noise=true , address=89.149.201.147:4014 , piece_id=73SJLJIBNXYUR43CRMDRD7XMGN6SVU27NEVYJRNXNK6CF4FXRPTA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:16.271862523 (190h16m12.952058092s) [noise=true , address=89.149.201.147:4001 , piece_id=LXKQNIY3I2MNN6YSJ2IIU5LHG7NDQTUYJV42BE3PF5ULUBT26RVA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:24:16.331658805 (190h16m13.011854384s) [noise=true , address=89.149.201.147:4017 , piece_id=OMREBLVGO4SHAJN4POZGOUCCIUYAFEYDPFNZRRHD7MTWWXZDWGKA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: <nil> Sep 6 12:25:13.689888835 (190h17m10.370084404s) [noise=true , address=185.167.204.206:4060 , piece_id=O7DYNS62TGN6EV2OBTSROK6WZW6UGBB3RUYNCVZH742DR3YJL4HA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 188.166.225.170:40488->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690343174 (190h17m10.370538743s) [noise=true , address=185.167.204.206:4066 , piece_id=PEOPIUS4XZTO4EBVAAYP2L76PLMOM4BE3ILXYO7YTOKUUOVWHNVQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: rpc: dial tcp 185.167.204.206:4066: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690371288 (190h17m10.370566857s) [noise=true , address=81.171.14.70:4040 , piece_id=W6RGCCK3TLDVSWZWMSCZDWOG7U7EP4K4MPM6TWRYJ6WMUXAHTVGA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:33640->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690404482 (190h17m10.370600051s) [noise=true , address=185.167.204.206:4071 , piece_id=4CSQ3CGS4UPCUZQQHOO3SB22CJDJDB5QVX57WGJKRYL3RU6B4CIQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: read tcp 188.166.225.170:43510->185.167.204.206:4071: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690415302 (190h17m10.370610881s) [noise=true , address=185.167.204.206:4040 , piece_id=T56QT5TDWKUXWKVTNDN76WQZRF45U2TKQA72DYGA7EU73D6HJ7XQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: rpc: dial tcp 185.167.204.206:4040: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690432285 (190h17m10.370627854s) [noise=true , address=185.167.204.205:4018 , piece_id=O4DUCO4FJSK63BYQ5ZQBMX5KHXX5H5JXEKXWSB2S5XRGDL36S5KQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:44406->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690462613 (190h17m10.370658172s) [noise=true , address=81.171.14.70:4029 , piece_id=4FKHKYWJUN5IDLQDG3ZLD75QF3VECUW66R3GQZLZ5UTSJKINKOAQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: read tcp 188.166.225.170:59458->81.171.14.70:4029: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690488372 (190h17m10.370683931s) [noise=true , address=185.167.204.206:4052 , piece_id=QYMJQA4UPMMAGDJ4ZKX4ITPIBY53MQ36DLT6HZGTVFIUHRDX42RA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12Xr3UmaAcshmxZMecu5i7VHA6LJxiZm9u6wngapEma3hz3tNmn, address:185.167.204.206:4052): protocol: read tcp 188.166.225.170:50568->185.167.204.206:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690515324 (190h17m10.370710893s) [noise=true , address=81.171.14.70:4034 , piece_id=3FEYWPM2HVXIFE3E25XP76XXD2KIELXDMAAUE6CH25K6MJGBI7IA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN, address:81.171.14.70:4034): protocol: read tcp 188.166.225.170:60868->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690544359 (190h17m10.370739928s) [noise=true , address=81.171.14.70:4046 , piece_id=ZFQC5POMGVVELAZVBENIZIXOIPL2JWITEB4FSNY63IUTBTJ4FCEA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:33170->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690570369 (190h17m10.370765928s) [noise=true , address=81.171.14.70:4026 , piece_id=SJGXNTQTBAN4TZLKE7UHSQATYXVIVOPKZKOUVQ4QJM7OBDO5WFSA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:35130->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690586990 (190h17m10.370782549s) [noise=true , address=81.171.14.70:4041 , piece_id=DTI55S76VI6RZWIUC5EJUO2DVY3PSYSDDGIZSQWXFT72B3AS5OUA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: rpc: dial tcp 81.171.14.70:4041: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690598834 (190h17m10.370794403s) [noise=true , address=81.171.14.70:4035 , piece_id=CG45UT6JBGNKKATFHETO4A2OK4DRRTVPEPPG6SR7PRDO6EXUZ5TQ, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 188.166.225.170:59518->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690616827 (190h17m10.370812396s) [noise=true , address=81.171.14.70:4045 , piece_id=DY2NZWZGKYO6DRE6UYLVYN5MHIYXZUWQCL6YGOTLYLU6X6WI6RAA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:41612->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690670831 (190h17m10.3708664s) [noise=true , address=185.167.204.206:4069 , piece_id=G5QMTCUEPC4OCQFDKCGDMRWNGL6LVQJEYVVN4TYP5LMIWU4RTXMA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: read tcp 188.166.225.170:41874->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690698444 (190h17m10.370894013s) [noise=true , address=185.167.204.206:4065 , piece_id=BQIBXKYI4V7NWYFYU4WXLTSXGALHTPR722344DDPH7C2LFC7HNMA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: read tcp 188.166.225.170:38262->185.167.204.206:4065: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690719264 (190h17m10.370914833s) [noise=true , address=185.167.204.206:4057 , piece_id=7TYEMHOGNRDWLVYDVKKHOQDQQJW4VXIPXGLF7IHW57U3KM2EKYWA, seg_pos={0 0}, upload=167411]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: read tcp 188.166.225.170:60604->185.167.204.206:4057: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:25:13.690739412 (190h17m10.370934982s) [seg_pos={0 0}, upload=167411]: Done waiting for segment. successful: 11 optimal: 26 errs: <nil> Sep 6 12:25:13.862966365 (190h17m10.543161934s) [upload=167411]: Done uploading segments... Sep 6 12:25:13.862994508 (190h17m10.543190077s) [upload=167411]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 11) Duration: 1m0.175597636s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000841272s FAIL (2025-09-06 12:25:12 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.000866089s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............X......X......................X......X.....X.X] duration: 30.001704551s FAIL (2025-09-06 12:25:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Sep 6 12:25:09.633908353 (190h17m6.314103922s) [upload=167420]: Starting upload Sep 6 12:25:09.633971915 (190h17m6.314167484s) [upload=167420]: Done starting upload Sep 6 12:25:09.634043041 (190h17m6.3142386s) [upload=167420]: Uploading segments... Sep 6 12:25:09.634145898 (190h17m6.314341467s) [upload=167420]: Got next segment. Inline: false Sep 6 12:25:09.634173571 (190h17m6.31436914s) [upload=167420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:25:09.634177739 (190h17m6.314373308s) [upload=167420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:25:11.949179651 (190h17m8.62937523s) [upload=167420]: Next segment err: context deadline exceeded Sep 6 12:25:11.949257410 (190h17m8.629452989s) [upload=167420]: Done uploading segments... Sep 6 12:25:11.949276818 (190h17m8.629472387s) [upload=167420]: Upload finished. err: context deadline exceeded Duration: 30.001725672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001773988s FAIL (2025-09-06 12:25:00 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 93454 (190h16m4.055289023s) [upload=167409]: Got next segment. Inline: false Sep 6 12:24:07.375119914 (190h16m4.055315483s) [upload=167409]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:24:07.375124783 (190h16m4.055320352s) [upload=167409]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:24:10.125893580 (190h16m6.806089149s) [seg_pos={0 0}, upload=167409]: Begin upload segment... Sep 6 12:24:10.125933998 (190h16m6.806129567s) [seg_pos={0 0}, upload=167409]: Done begin upload segment. Sep 6 12:24:10.125954537 (190h16m6.806150106s) [noise=true , address=77.90.31.21:4910 , piece_id=YUZMRI2PXHRKNQMSBNQHPJZOLANVZVMN7M3HXKT2OTCI5VY7Z5OQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.125963204 (190h16m6.806158773s) [noise=true , address=145.79.227.33:4872 , piece_id=BDNDKDD46U3UFFOQMJFL5KZSU7QWZQQ53N7VA6GMV5TI4LOAHHCA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.126315639 (190h16m6.806511208s) [noise=true , address=77.90.31.12:4596 , piece_id=RF5IB6DTH2ADFISG4LAHU22CJSBFZNX7ZJWBPFSJ4ULZYGZEMAYQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.126321920 (190h16m6.806517489s) [noise=true , address=77.90.31.18:4808 , piece_id=OQ6OS22WQBPJNAX3UMNTQSZU6LIE54BQHNGT4O37UUI5XLZGFHQQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.126687109 (190h16m6.806882678s) [noise=true , address=45.91.116.9:4278 , piece_id=3HERP4ATGZG2HDMCIAEUPRTIKLF7MFYYUATEQ6OOY5X5FP5E2PLA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.126881581 (190h16m6.80707717s) [noise=true , address=45.91.116.4:4946 , piece_id=LKQM2NMASRSDFAUEU3TT5QONTDQDPWEA5ZJ3F3YJBW3MX4IEOALA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.126984829 (190h16m6.807180398s) [noise=true , address=145.79.227.19:4091 , piece_id=7ATNPDTX3FSPZVFGHNGZAWKSHRD5N24LBKX76UOCEPBBNLJFIXOA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127222794 (190h16m6.807418363s) [noise=true , address=45.91.116.6:4007 , piece_id=XQWE3T2OVTYDKKT7V5XJIK3OTKCOOUL2IIHLE2X5O4H6YYI3KB3A, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127331803 (190h16m6.807527372s) [noise=true , address=45.91.116.10:4086 , piece_id=4KCOXK5APASLXGHLDQLJREAVN7RDB7RWDQYIEH6LVKE7ZC2UTOJQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127564708 (190h16m6.807760277s) [noise=true , address=145.79.227.49:4345 , piece_id=YCLTG7ADSICSQ7BUX5H4TQT6NWARHYGQ7EEYPMQVPGR7ZRZMSZJQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127602601 (190h16m6.80779817s) [noise=true , address=145.79.227.11:4734 , piece_id=TDA3CKGZXEJXPC2SS4L6664M2OS2JZE36UDI3LJIA5N3UI4QCLBQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127858250 (190h16m6.808053829s) [noise=true , address=77.90.31.15:4726 , piece_id=T4LCGXHB54AEDHK4UAA56QL257KZANA2YKVH22VYFW6E75TBP6DA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.127868429 (190h16m6.808063998s) [noise=true , address=77.90.31.14:4677 , piece_id=KSG7UOFKSNX2S2KAWIOPYXRZWB76LILQJDN4IZVOWO3Y5P7D6PQQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.128158995 (190h16m6.808354564s) [noise=true , address=145.79.227.44:4209 , piece_id=DGYRV22QXW2I3KGTWQG6QO3546V65BJSLUD2V3WVMFXT25LNKSEQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.128447207 (190h16m6.808642776s) [noise=true , address=145.79.227.37:4584 , piece_id=7BU6XTVXAE4QULYDWONYFNQEWAOYGMJ6EQRESKGZ42KDIBKPI77Q, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.128754505 (190h16m6.808950074s) [noise=true , address=145.79.227.16:4920 , piece_id=V2XQ7QVHPGGUAJ32SBMO3WRY4AEAQ7U4RI7QU4L6Y3UPSN7Y6BBQ, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.129009673 (190h16m6.809205242s) [noise=true , address=77.90.31.17:4782 , piece_id=VIJACP7EM5ARQOZSIXHQWQPBZJRRAGOTRVGQCPIX7WC3PNHZIXXA, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:10.130051527 (190h16m6.810247096s) [noise=true , address=145.79.227.14:4865 , piece_id=RMV5YZORSLW7NRZH4D3QKMBVDTNHMDLOR2GSJVHGD5XNP63LEK3Q, seg_pos={0 0}, upload=167409]: Uploading piece... Sep 6 12:24:11.677659174 (190h16m8.357854743s) [upload=167409]: Next returned nil segment Sep 6 12:24:11.677676026 (190h16m8.357871595s) [upload=167409]: Waiting for error group managing segments... Sep 6 12:24:12.106564529 (190h16m8.786760098s) [noise=true , address=145.79.227.44:4209 , piece_id=DGYRV22QXW2I3KGTWQG6QO3546V65BJSLUD2V3WVMFXT25LNKSEQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.123188980 (190h16m8.803384549s) [noise=true , address=45.91.116.6:4007 , piece_id=XQWE3T2OVTYDKKT7V5XJIK3OTKCOOUL2IIHLE2X5O4H6YYI3KB3A, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.132837873 (190h16m8.813033452s) [noise=true , address=77.90.31.14:4677 , piece_id=KSG7UOFKSNX2S2KAWIOPYXRZWB76LILQJDN4IZVOWO3Y5P7D6PQQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.150389519 (190h16m8.830585088s) [noise=true , address=145.79.227.49:4345 , piece_id=YCLTG7ADSICSQ7BUX5H4TQT6NWARHYGQ7EEYPMQVPGR7ZRZMSZJQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.152412340 (190h16m8.832607909s) [noise=true , address=145.79.227.19:4091 , piece_id=7ATNPDTX3FSPZVFGHNGZAWKSHRD5N24LBKX76UOCEPBBNLJFIXOA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.153006637 (190h16m8.833202206s) [noise=true , address=45.91.116.10:4086 , piece_id=4KCOXK5APASLXGHLDQLJREAVN7RDB7RWDQYIEH6LVKE7ZC2UTOJQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.155132615 (190h16m8.835328184s) [noise=true , address=77.90.31.18:4808 , piece_id=OQ6OS22WQBPJNAX3UMNTQSZU6LIE54BQHNGT4O37UUI5XLZGFHQQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.170368159 (190h16m8.850563718s) [noise=true , address=45.91.116.9:4278 , piece_id=3HERP4ATGZG2HDMCIAEUPRTIKLF7MFYYUATEQ6OOY5X5FP5E2PLA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.199360046 (190h16m8.879555615s) [noise=true , address=45.91.116.4:4946 , piece_id=LKQM2NMASRSDFAUEU3TT5QONTDQDPWEA5ZJ3F3YJBW3MX4IEOALA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.236374735 (190h16m8.916570304s) [noise=true , address=145.79.227.33:4872 , piece_id=BDNDKDD46U3UFFOQMJFL5KZSU7QWZQQ53N7VA6GMV5TI4LOAHHCA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.238424958 (190h16m8.918620527s) [noise=true , address=145.79.227.11:4734 , piece_id=TDA3CKGZXEJXPC2SS4L6664M2OS2JZE36UDI3LJIA5N3UI4QCLBQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.266441579 (190h16m8.946637148s) [noise=true , address=77.90.31.17:4782 , piece_id=VIJACP7EM5ARQOZSIXHQWQPBZJRRAGOTRVGQCPIX7WC3PNHZIXXA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.284381267 (190h16m8.964576836s) [noise=true , address=77.90.31.21:4910 , piece_id=YUZMRI2PXHRKNQMSBNQHPJZOLANVZVMN7M3HXKT2OTCI5VY7Z5OQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.302172150 (190h16m8.982367719s) [noise=true , address=77.90.31.15:4726 , piece_id=T4LCGXHB54AEDHK4UAA56QL257KZANA2YKVH22VYFW6E75TBP6DA, seg_pos={0 0}, upload=167409]: Done uploading piece. err: <nil> Sep 6 12:24:12.302197629 (190h16m8.982393198s) [seg_pos={0 0}, upload=167409]: Segment reached optimal threshold of 14 pieces. Sep 6 12:24:12.302286880 (190h16m8.982482449s) [noise=true , address=145.79.227.37:4584 , piece_id=7BU6XTVXAE4QULYDWONYFNQEWAOYGMJ6EQRESKGZ42KDIBKPI77Q, seg_pos={0 0}, upload=167409]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S): protocol: expected piece hash; context canceled; EOF Sep 6 12:24:12.302327317 (190h16m8.982522886s) [noise=true , address=145.79.227.16:4920 , piece_id=V2XQ7QVHPGGUAJ32SBMO3WRY4AEAQ7U4RI7QU4L6Y3UPSN7Y6BBQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vBccAgciQwbLaMMe9QKfYM3zxYTHqG3pfYQ7nh8eETr8JUskb): protocol: expected piece hash; context canceled; EOF Sep 6 12:24:12.302455863 (190h16m8.982651432s) [noise=true , address=77.90.31.12:4596 , piece_id=RF5IB6DTH2ADFISG4LAHU22CJSBFZNX7ZJWBPFSJ4ULZYGZEMAYQ, seg_pos={0 0}, upload=167409]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf): protocol: expected piece hash; context canceled; EOF Sep 6 12:24:12.302484899 (190h16m8.982680468s) [noise=true , address=145.79.227.14:4865 , piece_id=RMV5YZORSLW7NRZH4D3QKMBVDTNHMDLOR2GSJVHGD5XNP63LEK3Q, seg_pos={0 0}, upload=167409]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kqNGahsBw6VsV3JTALvPyGshuSFaAdRSo94cQnCXF3ak54KtL): protocol: write tcp 188.166.225.170:52470->145.79.227.14:4865: use of closed network connection; write tcp 188.166.225.170:52470->145.79.227.14:4865: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52470->145.79.227.14:4865: use of closed network connection Sep 6 12:24:12.302507502 (190h16m8.982703071s) [seg_pos={0 0}, upload=167409]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 6 12:24:12.302568879 (190h16m8.982764448s) [upload=167409]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:24:12.302572556 (190h16m8.982768125s) [upload=167409]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:24:12.716822488 (190h16m9.397018057s) [upload=167409]: Done uploading segments... Sep 6 12:24:12.716844148 (190h16m9.397039708s) [upload=167409]: Upload finished. err: <nil> Duration: 1m0.001798064s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001351607s FAIL (2025-09-06 12:24:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001374619s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X......X......................X......X.....X.] duration: 16.660433606s FAIL (2025-09-06 12:24:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:22:42.381648685 (190h14m39.061844244s) [upload=167389]: Starting upload Sep 6 12:22:42.381718910 (190h14m39.061914479s) [upload=167389]: Done starting upload Sep 6 12:22:42.381834331 (190h14m39.0620299s) [upload=167389]: Uploading segments... Sep 6 12:22:42.381958047 (190h14m39.062153616s) [upload=167389]: Got next segment. Inline: false Sep 6 12:22:42.381998655 (190h14m39.062194224s) [upload=167389]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:22:42.382003194 (190h14m39.062198763s) [upload=167389]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:22:44.202626566 (190h14m40.882822135s) [seg_pos={0 0}, upload=167389]: Begin upload segment... Sep 6 12:22:44.202684698 (190h14m40.882880267s) [seg_pos={0 0}, upload=167389]: Done begin upload segment. Sep 6 12:22:44.202711970 (190h14m40.882907539s) [noise=true , address=77.90.31.21:4879 , piece_id=IWUB3OZGS7BAWLDO3FKUYREYUPDDMQWNGKVM7VWJSIIRA3YY54IQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.202721939 (190h14m40.882917528s) [noise=true , address=77.90.31.18:4807 , piece_id=LWT422BHNCMCTRGL37VKO4B3HISVIUVOONYIURNWG65JWIDOUN4A, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203116974 (190h14m40.883312554s) [noise=true , address=45.91.116.9:4263 , piece_id=ZAMFAP6JVDKTT65ZRMWTWN3EJ2DQC5VRGN72RCDYISRSBRNDWIXQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203257523 (190h14m40.883453082s) [noise=true , address=77.90.31.21:4878 , piece_id=L3LUTZCZQBP652GWRVB4E3HOEMTATEZUFBGO6NGQMSWZ7UP7ORSQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203466644 (190h14m40.883662213s) [noise=true , address=45.91.116.6:4974 , piece_id=LYKHNHKROTJLD5KYA3MHJ5CRK34APJF2LDUQMZLTB5AN5JAKTEWA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203519004 (190h14m40.883714563s) [noise=true , address=77.90.31.12:4590 , piece_id=ETDM2AUDZD2ZYY5YD3E5WTPFN32QOXRWDOWECZD45QGBY7JL7EHA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203764784 (190h14m40.883960353s) [noise=true , address=45.91.116.12:4225 , piece_id=Y5KQDQHCMKC3YHFE3RQKWMVOYXVARSPE2QA3USK5HYVAHME4HNEQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203792958 (190h14m40.883988527s) [noise=true , address=45.91.116.14:4189 , piece_id=3YTLXB6RVMJSIKTTVFPEBRHTCQU35KTJHKRZJRGFL2AXSDUUJZKQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204091510 (190h14m40.884287079s) [noise=true , address=45.91.116.8:4048 , piece_id=DKZ7QPEBSGQ6NXI2C5V4YHQ677CYEHODGD25ORA5Z45LNNV4SZTA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204304277 (190h14m40.884499846s) [noise=true , address=45.91.116.10:4081 , piece_id=ADSKUSSQRSAM3TKJFOZ7PRKUDHJVZ6W6HARH33UNAQ4PW6UPL7QA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204530660 (190h14m40.884726249s) [noise=true , address=45.91.116.6:4973 , piece_id=VSNF5ASVGA5ASGDLRDGDOI26CVX4HAKRBGW6DVXIRIEDESVI74FA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204844761 (190h14m40.88504033s) [noise=true , address=45.91.116.13:4154 , piece_id=LHZMM2P7NIHAM4W27N53K4QCQLSFD2EXOXLKEOGKHL42BXHN7KVA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205159082 (190h14m40.885354651s) [noise=true , address=77.90.31.14:4663 , piece_id=LS4FYSH362AZAM2N6PR7GAWNV52FSB3NXUPIHC2WB7HHDX4FU2KA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205358363 (190h14m40.885553932s) [noise=true , address=45.91.116.10:4082 , piece_id=XE2MGTI3LSI66QJN4LSBYIFDSDKBASZMGA5A6REEZ2P6F435SH7Q, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205550882 (190h14m40.885746451s) [noise=true , address=45.91.116.11:4117 , piece_id=FY2X4HSKPQMPKTJ6E3B4I2H5VZH5YXIMLSWLURFZLGNSALUEDPUA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.206689581 (190h14m40.88688515s) [noise=true , address=45.91.116.13:4153 , piece_id=5FW3W463HXQSGXAJN252ZHAS52VUL6SCEOST4IVMHSP7O3VUOPLQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:46.012766398 (190h14m42.692961967s) [upload=167389]: Next returned nil segment Sep 6 12:22:46.012783211 (190h14m42.69297878s) [upload=167389]: Waiting for error group managing segments... Sep 6 12:22:46.370598261 (190h14m43.05079384s) [noise=true , address=45.91.116.11:4117 , piece_id=FY2X4HSKPQMPKTJ6E3B4I2H5VZH5YXIMLSWLURFZLGNSALUEDPUA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.371457606 (190h14m43.051653185s) [noise=true , address=45.91.116.8:4048 , piece_id=DKZ7QPEBSGQ6NXI2C5V4YHQ677CYEHODGD25ORA5Z45LNNV4SZTA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.372858987 (190h14m43.053054556s) [noise=true , address=45.91.116.6:4974 , piece_id=LYKHNHKROTJLD5KYA3MHJ5CRK34APJF2LDUQMZLTB5AN5JAKTEWA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.373429639 (190h14m43.053625208s) [noise=true , address=45.91.116.14:4189 , piece_id=3YTLXB6RVMJSIKTTVFPEBRHTCQU35KTJHKRZJRGFL2AXSDUUJZKQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.375182423 (190h14m43.055377992s) [noise=true , address=45.91.116.10:4082 , piece_id=XE2MGTI3LSI66QJN4LSBYIFDSDKBASZMGA5A6REEZ2P6F435SH7Q, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.382850005 (190h14m43.063045574s) [noise=true , address=77.90.31.21:4879 , piece_id=IWUB3OZGS7BAWLDO3FKUYREYUPDDMQWNGKVM7VWJSIIRA3YY54IQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.389768875 (190h14m43.069964444s) [noise=true , address=45.91.116.10:4081 , piece_id=ADSKUSSQRSAM3TKJFOZ7PRKUDHJVZ6W6HARH33UNAQ4PW6UPL7QA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.394024998 (190h14m43.074220567s) [noise=true , address=45.91.116.6:4973 , piece_id=VSNF5ASVGA5ASGDLRDGDOI26CVX4HAKRBGW6DVXIRIEDESVI74FA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.410606408 (190h14m43.090801977s) [noise=true , address=45.91.116.13:4153 , piece_id=5FW3W463HXQSGXAJN252ZHAS52VUL6SCEOST4IVMHSP7O3VUOPLQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.419207085 (190h14m43.099402654s) [noise=true , address=45.91.116.9:4263 , piece_id=ZAMFAP6JVDKTT65ZRMWTWN3EJ2DQC5VRGN72RCDYISRSBRNDWIXQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.487672296 (190h14m43.167867865s) [noise=true , address=77.90.31.14:4663 , piece_id=LS4FYSH362AZAM2N6PR7GAWNV52FSB3NXUPIHC2WB7HHDX4FU2KA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.515482662 (190h14m43.195678231s) [noise=true , address=77.90.31.21:4878 , piece_id=L3LUTZCZQBP652GWRVB4E3HOEMTATEZUFBGO6NGQMSWZ7UP7ORSQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.542680667 (190h14m43.222876246s) [noise=true , address=77.90.31.18:4807 , piece_id=LWT422BHNCMCTRGL37VKO4B3HISVIUVOONYIURNWG65JWIDOUN4A, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.542708460 (190h14m43.222904039s) [seg_pos={0 0}, upload=167389]: Segment reached optimal threshold of 13 pieces. Sep 6 12:22:46.543145025 (190h14m43.223340594s) [noise=true , address=77.90.31.12:4590 , piece_id=ETDM2AUDZD2ZYY5YD3E5WTPFN32QOXRWDOWECZD45QGBY7JL7EHA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 6 12:22:46.543174622 (190h14m43.223370191s) [noise=true , address=45.91.116.12:4225 , piece_id=Y5KQDQHCMKC3YHFE3RQKWMVOYXVARSPE2QA3USK5HYVAHME4HNEQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection Sep 6 12:22:46.543314439 (190h14m43.223510008s) [noise=true , address=45.91.116.13:4154 , piece_id=LHZMM2P7NIHAM4W27N53K4QCQLSFD2EXOXLKEOGKHL42BXHN7KVA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection; write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection Sep 6 12:22:46.543363654 (190h14m43.223559223s) [seg_pos={0 0}, upload=167389]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:22:46.543452213 (190h14m43.223647782s) [upload=167389]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:22:46.543457593 (190h14m43.223653172s) [upload=167389]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:22:46.890768985 (190h14m43.570964554s) [upload=167389]: Done uploading segments... Sep 6 12:22:46.890788472 (190h14m43.570984041s) [upload=167389]: Upload finished. err: <nil> Duration: 16.660462811s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X......X......................X......X.....X] duration: 30.002428189s FAIL (2025-09-06 12:23:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:22:42.381648685 (190h14m39.061844244s) [upload=167389]: Starting upload Sep 6 12:22:42.381718910 (190h14m39.061914479s) [upload=167389]: Done starting upload Sep 6 12:22:42.381834331 (190h14m39.0620299s) [upload=167389]: Uploading segments... Sep 6 12:22:42.381958047 (190h14m39.062153616s) [upload=167389]: Got next segment. Inline: false Sep 6 12:22:42.381998655 (190h14m39.062194224s) [upload=167389]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:22:42.382003194 (190h14m39.062198763s) [upload=167389]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:22:44.202626566 (190h14m40.882822135s) [seg_pos={0 0}, upload=167389]: Begin upload segment... Sep 6 12:22:44.202684698 (190h14m40.882880267s) [seg_pos={0 0}, upload=167389]: Done begin upload segment. Sep 6 12:22:44.202711970 (190h14m40.882907539s) [noise=true , address=77.90.31.21:4879 , piece_id=IWUB3OZGS7BAWLDO3FKUYREYUPDDMQWNGKVM7VWJSIIRA3YY54IQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.202721939 (190h14m40.882917528s) [noise=true , address=77.90.31.18:4807 , piece_id=LWT422BHNCMCTRGL37VKO4B3HISVIUVOONYIURNWG65JWIDOUN4A, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203116974 (190h14m40.883312554s) [noise=true , address=45.91.116.9:4263 , piece_id=ZAMFAP6JVDKTT65ZRMWTWN3EJ2DQC5VRGN72RCDYISRSBRNDWIXQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203257523 (190h14m40.883453082s) [noise=true , address=77.90.31.21:4878 , piece_id=L3LUTZCZQBP652GWRVB4E3HOEMTATEZUFBGO6NGQMSWZ7UP7ORSQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203466644 (190h14m40.883662213s) [noise=true , address=45.91.116.6:4974 , piece_id=LYKHNHKROTJLD5KYA3MHJ5CRK34APJF2LDUQMZLTB5AN5JAKTEWA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203519004 (190h14m40.883714563s) [noise=true , address=77.90.31.12:4590 , piece_id=ETDM2AUDZD2ZYY5YD3E5WTPFN32QOXRWDOWECZD45QGBY7JL7EHA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203764784 (190h14m40.883960353s) [noise=true , address=45.91.116.12:4225 , piece_id=Y5KQDQHCMKC3YHFE3RQKWMVOYXVARSPE2QA3USK5HYVAHME4HNEQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.203792958 (190h14m40.883988527s) [noise=true , address=45.91.116.14:4189 , piece_id=3YTLXB6RVMJSIKTTVFPEBRHTCQU35KTJHKRZJRGFL2AXSDUUJZKQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204091510 (190h14m40.884287079s) [noise=true , address=45.91.116.8:4048 , piece_id=DKZ7QPEBSGQ6NXI2C5V4YHQ677CYEHODGD25ORA5Z45LNNV4SZTA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204304277 (190h14m40.884499846s) [noise=true , address=45.91.116.10:4081 , piece_id=ADSKUSSQRSAM3TKJFOZ7PRKUDHJVZ6W6HARH33UNAQ4PW6UPL7QA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204530660 (190h14m40.884726249s) [noise=true , address=45.91.116.6:4973 , piece_id=VSNF5ASVGA5ASGDLRDGDOI26CVX4HAKRBGW6DVXIRIEDESVI74FA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.204844761 (190h14m40.88504033s) [noise=true , address=45.91.116.13:4154 , piece_id=LHZMM2P7NIHAM4W27N53K4QCQLSFD2EXOXLKEOGKHL42BXHN7KVA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205159082 (190h14m40.885354651s) [noise=true , address=77.90.31.14:4663 , piece_id=LS4FYSH362AZAM2N6PR7GAWNV52FSB3NXUPIHC2WB7HHDX4FU2KA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205358363 (190h14m40.885553932s) [noise=true , address=45.91.116.10:4082 , piece_id=XE2MGTI3LSI66QJN4LSBYIFDSDKBASZMGA5A6REEZ2P6F435SH7Q, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.205550882 (190h14m40.885746451s) [noise=true , address=45.91.116.11:4117 , piece_id=FY2X4HSKPQMPKTJ6E3B4I2H5VZH5YXIMLSWLURFZLGNSALUEDPUA, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:44.206689581 (190h14m40.88688515s) [noise=true , address=45.91.116.13:4153 , piece_id=5FW3W463HXQSGXAJN252ZHAS52VUL6SCEOST4IVMHSP7O3VUOPLQ, seg_pos={0 0}, upload=167389]: Uploading piece... Sep 6 12:22:46.012766398 (190h14m42.692961967s) [upload=167389]: Next returned nil segment Sep 6 12:22:46.012783211 (190h14m42.69297878s) [upload=167389]: Waiting for error group managing segments... Sep 6 12:22:46.370598261 (190h14m43.05079384s) [noise=true , address=45.91.116.11:4117 , piece_id=FY2X4HSKPQMPKTJ6E3B4I2H5VZH5YXIMLSWLURFZLGNSALUEDPUA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.371457606 (190h14m43.051653185s) [noise=true , address=45.91.116.8:4048 , piece_id=DKZ7QPEBSGQ6NXI2C5V4YHQ677CYEHODGD25ORA5Z45LNNV4SZTA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.372858987 (190h14m43.053054556s) [noise=true , address=45.91.116.6:4974 , piece_id=LYKHNHKROTJLD5KYA3MHJ5CRK34APJF2LDUQMZLTB5AN5JAKTEWA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.373429639 (190h14m43.053625208s) [noise=true , address=45.91.116.14:4189 , piece_id=3YTLXB6RVMJSIKTTVFPEBRHTCQU35KTJHKRZJRGFL2AXSDUUJZKQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.375182423 (190h14m43.055377992s) [noise=true , address=45.91.116.10:4082 , piece_id=XE2MGTI3LSI66QJN4LSBYIFDSDKBASZMGA5A6REEZ2P6F435SH7Q, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.382850005 (190h14m43.063045574s) [noise=true , address=77.90.31.21:4879 , piece_id=IWUB3OZGS7BAWLDO3FKUYREYUPDDMQWNGKVM7VWJSIIRA3YY54IQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.389768875 (190h14m43.069964444s) [noise=true , address=45.91.116.10:4081 , piece_id=ADSKUSSQRSAM3TKJFOZ7PRKUDHJVZ6W6HARH33UNAQ4PW6UPL7QA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.394024998 (190h14m43.074220567s) [noise=true , address=45.91.116.6:4973 , piece_id=VSNF5ASVGA5ASGDLRDGDOI26CVX4HAKRBGW6DVXIRIEDESVI74FA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.410606408 (190h14m43.090801977s) [noise=true , address=45.91.116.13:4153 , piece_id=5FW3W463HXQSGXAJN252ZHAS52VUL6SCEOST4IVMHSP7O3VUOPLQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.419207085 (190h14m43.099402654s) [noise=true , address=45.91.116.9:4263 , piece_id=ZAMFAP6JVDKTT65ZRMWTWN3EJ2DQC5VRGN72RCDYISRSBRNDWIXQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.487672296 (190h14m43.167867865s) [noise=true , address=77.90.31.14:4663 , piece_id=LS4FYSH362AZAM2N6PR7GAWNV52FSB3NXUPIHC2WB7HHDX4FU2KA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.515482662 (190h14m43.195678231s) [noise=true , address=77.90.31.21:4878 , piece_id=L3LUTZCZQBP652GWRVB4E3HOEMTATEZUFBGO6NGQMSWZ7UP7ORSQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.542680667 (190h14m43.222876246s) [noise=true , address=77.90.31.18:4807 , piece_id=LWT422BHNCMCTRGL37VKO4B3HISVIUVOONYIURNWG65JWIDOUN4A, seg_pos={0 0}, upload=167389]: Done uploading piece. err: <nil> Sep 6 12:22:46.542708460 (190h14m43.222904039s) [seg_pos={0 0}, upload=167389]: Segment reached optimal threshold of 13 pieces. Sep 6 12:22:46.543145025 (190h14m43.223340594s) [noise=true , address=77.90.31.12:4590 , piece_id=ETDM2AUDZD2ZYY5YD3E5WTPFN32QOXRWDOWECZD45QGBY7JL7EHA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 6 12:22:46.543174622 (190h14m43.223370191s) [noise=true , address=45.91.116.12:4225 , piece_id=Y5KQDQHCMKC3YHFE3RQKWMVOYXVARSPE2QA3USK5HYVAHME4HNEQ, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53604->45.91.116.12:4225: use of closed network connection Sep 6 12:22:46.543314439 (190h14m43.223510008s) [noise=true , address=45.91.116.13:4154 , piece_id=LHZMM2P7NIHAM4W27N53K4QCQLSFD2EXOXLKEOGKHL42BXHN7KVA, seg_pos={0 0}, upload=167389]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection; write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48184->45.91.116.13:4154: use of closed network connection Sep 6 12:22:46.543363654 (190h14m43.223559223s) [seg_pos={0 0}, upload=167389]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:22:46.543452213 (190h14m43.223647782s) [upload=167389]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:22:46.543457593 (190h14m43.223653172s) [upload=167389]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:22:46.890768985 (190h14m43.570964554s) [upload=167389]: Done uploading segments... Sep 6 12:22:46.890788472 (190h14m43.570984041s) [upload=167389]: Upload finished. err: <nil> Duration: 30.002455852s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X......X......................X......X.....] duration: 6.400297489s FAIL (2025-09-06 12:21:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 6.400323619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X......X......................X......X....] duration: 29.005005726s FAIL (2025-09-06 12:21:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 29.005024091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X......X......................X......X...] duration: 24.406188421s FAIL (2025-09-06 12:20:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 24.406213559s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X......X......................X......X..] duration: 6.363933977s FAIL (2025-09-06 12:20:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 6.363959698s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X......X......................X......X.] duration: 7.994923873s FAIL (2025-09-06 12:19:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 7.994955032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X......X......................X......X] duration: 30.002470234s FAIL (2025-09-06 12:18:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:18:25.834180501 (190h10m22.514376071s) [upload=167325]: Starting upload Sep 6 12:18:25.834242330 (190h10m22.514437889s) [upload=167325]: Done starting upload Sep 6 12:18:25.834320129 (190h10m22.514515698s) [upload=167325]: Uploading segments... Sep 6 12:18:25.834447021 (190h10m22.51464263s) [upload=167325]: Got next segment. Inline: false Sep 6 12:18:25.834497768 (190h10m22.514693337s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:18:25.834504441 (190h10m22.51470001s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:18:26.112913485 (190h10m22.793109054s) [seg_pos={0 0}, upload=167325]: Begin upload segment... Sep 6 12:18:26.112953142 (190h10m22.793148711s) [seg_pos={0 0}, upload=167325]: Done begin upload segment. Sep 6 12:18:26.112976186 (190h10m22.793171755s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.112982888 (190h10m22.793178458s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113384597 (190h10m22.793580167s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113666196 (190h10m22.793861755s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.113969386 (190h10m22.794164955s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114045883 (190h10m22.794241502s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114378018 (190h10m22.794573587s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.114599101 (190h10m22.79479467s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115187237 (190h10m22.795382796s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115388472 (190h10m22.795584041s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.115803095 (190h10m22.795998664s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116076789 (190h10m22.796272358s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.116955019 (190h10m22.797150588s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117201410 (190h10m22.797396979s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117441410 (190h10m22.797636979s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:26.117669165 (190h10m22.797864734s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Uploading piece... Sep 6 12:18:27.925177143 (190h10m24.605372712s) [upload=167325]: Next returned nil segment Sep 6 12:18:27.925239692 (190h10m24.605435271s) [upload=167325]: Waiting for error group managing segments... Sep 6 12:18:28.210980757 (190h10m24.891176327s) [noise=true , address=45.91.116.10:4082 , piece_id=ZFQLEXDDGW4B2WYZIAT5ITQWMTUBW7J2NYKERKEZEOUDSFLGHX4Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.242221678 (190h10m24.922417267s) [noise=true , address=45.91.116.13:4154 , piece_id=JEE5SGP5IPD5IUJV32GQVGD7BMNOI5EZIVZUFPI6IRRYKPSDFUGQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.262058696 (190h10m24.942254265s) [noise=true , address=45.91.116.5:4298 , piece_id=KAPZH6VEVGLH45NVOYQ7CRHV4PSNZPHBVB73FBWN6D2EBB7GO7JQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.270305918 (190h10m24.950501487s) [noise=true , address=45.91.116.11:4117 , piece_id=F7DCX2CHFEGNNW73JZSZQVC7RHG7R3PK4QO3DBKIEQFFDWLN5R6A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.272851700 (190h10m24.953047269s) [noise=true , address=45.91.116.6:4974 , piece_id=3EARMVDK5N6T4MPQVPXU2WZAAF4AZHUAUTXYZUF56TSQ2TXFM62Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.279616574 (190h10m24.959812143s) [noise=true , address=77.90.31.17:4771 , piece_id=GXDWTHKCAMDLK3PVHSVXUD7QHY5GXJ7C6KXQDCP55K4FH6ZE7ENQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.283376800 (190h10m24.963572359s) [noise=true , address=45.91.116.12:4225 , piece_id=U5NCOQ3TGJH3CS66D4JC5CPBESEEREFMQOJPS4PXG2GFCBEL3JKQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.299802300 (190h10m24.979997869s) [noise=true , address=45.91.116.4:4937 , piece_id=BLNSO3KFKGGKD4M5QFCFRJPRXN6QFRVXTSZA4B4PGWKM3LJLRK2Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.309231183 (190h10m24.989426742s) [noise=true , address=45.91.116.4:4939 , piece_id=YNYJDK74OQVNSNB5DJPJYZI2P6RRVO2ETNFVKOZCVIDFSHQPGWSA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.317529011 (190h10m24.99772458s) [noise=true , address=45.91.116.5:4297 , piece_id=BHGCPN3IG2U5MYM7DLHVXIIR7H6LSLKWPGULVVBQO6FI3XBXCP7Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.326559361 (190h10m25.00675493s) [noise=true , address=45.91.116.8:4047 , piece_id=4XHMXUY2ENZNE2YXICCUZP5AIMU6HOCPMDJVSSXIX2SH7YDWMJFA, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.347489141 (190h10m25.02768471s) [noise=true , address=45.91.116.4:4938 , piece_id=MAGRSIZXYR6IKQJHNF6V2S55YBNQ76WZK7EMX6NY7UDZDMWPJGCQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353529009 (190h10m25.033724578s) [noise=true , address=45.91.116.9:4263 , piece_id=ASR7WYGIWSN6PSGG4MM3J56ITSXSSB44WTNMKIC6NTXMLBDKTN2A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: <nil> Sep 6 12:18:28.353548004 (190h10m25.033743573s) [seg_pos={0 0}, upload=167325]: Segment reached optimal threshold of 13 pieces. Sep 6 12:18:28.353655260 (190h10m25.033850829s) [noise=true , address=45.91.116.6:4973 , piece_id=TCSXTEKCRY4OZUND6IELMLRLC4VTWWAR6MCBWMFUJRQ2554MV64A, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353734732 (190h10m25.033930301s) [noise=true , address=77.90.31.19:4843 , piece_id=UIIJI3J543WJXEFV2ME2VDPIM232CH3AFCEK4YOJUXHC5H2KMYJQ, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353787803 (190h10m25.033983372s) [noise=true , address=45.91.116.14:4191 , piece_id=2VW3F4KOOSFCNIGP3FOSGKE7N2FA3HKK4A7HAT2X3QL6MQQBMJ6Q, seg_pos={0 0}, upload=167325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 12:18:28.353815717 (190h10m25.034011286s) [seg_pos={0 0}, upload=167325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:18:28.353874549 (190h10m25.034070118s) [upload=167325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:18:28.353878467 (190h10m25.034074036s) [upload=167325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:18:28.685851017 (190h10m25.366046586s) [upload=167325]: Done uploading segments... Sep 6 12:18:28.685870794 (190h10m25.366066363s) [upload=167325]: Upload finished. err: <nil> Duration: 30.002491686s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X......X......................X......] duration: 7.063636763s FAIL (2025-09-06 12:18:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 7.063661771s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X......X......................X.....] duration: 6.279856425s FAIL (2025-09-06 12:16:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 6.279873629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X......X......................X....] duration: 6.66636287s FAIL (2025-09-06 12:16:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 6.666391916s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X......X......................X...] duration: 6.360502893s FAIL (2025-09-06 12:14:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 6.360525364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X......X......................X..] duration: 7.213346671s FAIL (2025-09-06 12:14:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 7.21337216s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X......X......................X.] duration: 11.605581987s FAIL (2025-09-06 12:14:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 11.605607045s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X......X......................X] duration: 30.00151663s FAIL (2025-09-06 12:13:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 12:13:08.495030447 (190h5m5.175226016s) [upload=167242]: Starting upload Sep 6 12:13:08.495144416 (190h5m5.175339985s) [upload=167242]: Done starting upload Sep 6 12:13:08.495228978 (190h5m5.175424547s) [upload=167242]: Uploading segments... Sep 6 12:13:08.495390938 (190h5m5.175586507s) [upload=167242]: Got next segment. Inline: false Sep 6 12:13:08.495422769 (190h5m5.175618338s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 12:13:08.495427708 (190h5m5.175623267s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 12:13:08.973055064 (190h5m5.653250633s) [seg_pos={0 0}, upload=167242]: Begin upload segment... Sep 6 12:13:08.973095211 (190h5m5.65329078s) [seg_pos={0 0}, upload=167242]: Done begin upload segment. Sep 6 12:13:08.973122032 (190h5m5.653317601s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973606880 (190h5m5.653802449s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973631437 (190h5m5.653827006s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.973896784 (190h5m5.654092353s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974096006 (190h5m5.654291575s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974238018 (190h5m5.654433587s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974389999 (190h5m5.654585568s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974545446 (190h5m5.654741015s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974705743 (190h5m5.654901302s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974791236 (190h5m5.654986805s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.974968866 (190h5m5.655164435s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975703552 (190h5m5.655899121s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.975716577 (190h5m5.655912146s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.976530124 (190h5m5.656725693s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.982703236 (190h5m5.662898805s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:08.986247177 (190h5m5.666442746s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Uploading piece... Sep 6 12:13:10.813365930 (190h5m7.493561499s) [upload=167242]: Next returned nil segment Sep 6 12:13:10.813383804 (190h5m7.493579373s) [upload=167242]: Waiting for error group managing segments... Sep 6 12:13:11.117375092 (190h5m7.797570661s) [noise=true , address=45.91.116.5:4297 , piece_id=JMBZTYBVUICHH2M6TAEG2KXL6K6VGXTFU3KVEIEIV2ZIU7MYEZKA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.131314225 (190h5m7.811509794s) [noise=true , address=45.91.116.5:4299 , piece_id=BJBX4LHMJYM23VPNNI2CENUUSYTTXAZLGJFMSZJVSGZ6LIGC7F3Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.145539966 (190h5m7.825735535s) [noise=true , address=45.91.116.10:4082 , piece_id=PCBAZMTEFQHOGJ24WMTMB44URZ7AGAUULUBORSTEROEUKB2AIMCQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.146303537 (190h5m7.826499117s) [noise=true , address=45.91.116.8:4047 , piece_id=NZNNW6X6AH6FHNMV3Z6BO7OQUK4JMEO3NBO6QPBEV4PBDNV7ICBA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.163692451 (190h5m7.84388802s) [noise=true , address=45.91.116.4:4939 , piece_id=APSO2JGMGQIDPNH7DOXWRBZ2RA7JT3DDAKNG4RI2BGPS3MRZG4WA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.164112004 (190h5m7.844307573s) [noise=true , address=45.91.116.6:4974 , piece_id=3OQZ2THNRTX3FU6BVH2GYMOVTPTIGYE3AOJETXX6SRPFBBJ7D6HQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.185097882 (190h5m7.865293451s) [noise=true , address=45.91.116.13:4154 , piece_id=CCL6EKC2ZD2V5JXIEM6V2OKNBAE5M7AI6YX75XJP5FO3EFI3R5JQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.189347723 (190h5m7.869543293s) [noise=true , address=77.90.31.10:4517 , piece_id=JSHPMSIDUC2BCKUL2VEKM7WNBRIXZZC5MZOMEVXPRTG2NT7JMP3A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.244755157 (190h5m7.924950726s) [noise=true , address=45.91.116.10:4081 , piece_id=5MN7TAD2KIHWH7LJQAPJAE4VVYDCPKCFXNJLOILKRPA2HRRDSDXQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.248405924 (190h5m7.928601493s) [noise=true , address=77.90.31.19:4843 , piece_id=E3OAOVSCUORFYKOVK6AGHB63AGY7BOWINO4XFSIWXIYU3HXBLIUA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.255020320 (190h5m7.935215889s) [noise=true , address=45.91.116.14:4191 , piece_id=LTPKXBOHL64SPJK5MWZGQF4O4JNPZNPBDOFRMVJTDEMERROGTJGQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.282885211 (190h5m7.96308078s) [noise=true , address=45.91.116.7:4011 , piece_id=KRRVSOIIEVM3G43OIKB2FRD4HG2YWWVGIU6K7GALFHBPRL6LDPZA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291758791 (190h5m7.97195437s) [noise=true , address=45.91.116.6:4973 , piece_id=D6M22PV2EENOHB5YY54O6WUMPTDY4IPGWMZO4DDYIQN6HLJBIO5Q, seg_pos={0 0}, upload=167242]: Done uploading piece. err: <nil> Sep 6 12:13:11.291785141 (190h5m7.97198071s) [seg_pos={0 0}, upload=167242]: Segment reached optimal threshold of 13 pieces. Sep 6 12:13:11.291900963 (190h5m7.972096522s) [noise=true , address=45.91.116.13:4153 , piece_id=TNHVIBPVIC6RVPFQFQXZUZLQQWPVZ2KXUFGN6QDUUBWZOK5MIFJQ, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 6 12:13:11.292053034 (190h5m7.972248613s) [noise=true , address=77.90.31.17:4770 , piece_id=2DX54XIGEABNA2RNP5T3M35C327FBU7LGSICTQW5LNPQDODTJHIA, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57284->77.90.31.17:4770: use of closed network connection Sep 6 12:13:11.292099012 (190h5m7.972294581s) [noise=true , address=45.91.116.11:4117 , piece_id=YICDKVYCBAXVLBZUXKHTE3VKNNVRCLZXFF4K3YNDSC7PXFS65H4A, seg_pos={0 0}, upload=167242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: read tcp 188.166.225.170:46990->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 12:13:11.292121064 (190h5m7.972316643s) [seg_pos={0 0}, upload=167242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 12:13:11.292197651 (190h5m7.97239323s) [upload=167242]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 12:13:11.292201728 (190h5m7.972397297s) [upload=167242]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 12:13:11.621472490 (190h5m8.301668059s) [upload=167242]: Done uploading segments... Sep 6 12:13:11.621491846 (190h5m8.301687415s) [upload=167242]: Upload finished. err: <nil> Duration: 30.001537492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X......X......................] duration: 7.557831312s FAIL (2025-09-06 12:12:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 7.557857511s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X......X.....................] duration: 7.644675461s FAIL (2025-09-06 12:11:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 7.644699646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X......X....................] duration: 10.29208727s FAIL (2025-09-06 12:10:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 10.292113241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X......X...................] duration: 11.032522089s FAIL (2025-09-06 12:09:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 11.032547536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X......X..................] duration: 9.528976736s FAIL (2025-09-06 12:09:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 9.529009258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..................................X......X.................] duration: 10.667148152s FAIL (2025-09-06 12:08:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 10.667165015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X..................................X......X................] duration: 18.859998172s FAIL (2025-09-06 12:07:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 18.860028188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X..................................X......X...............] duration: 15.73032445s FAIL (2025-09-06 12:07:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 15.730337856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..................................X......X..............] duration: 6.793579399s FAIL (2025-09-06 12:06:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 11:58:24.744061884 (189h50m21.424257483s) [upload=167027]: Starting upload Sep 6 11:58:24.744178016 (189h50m21.424373625s) [upload=167027]: Done starting upload Sep 6 11:58:24.744278228 (189h50m21.424473827s) [upload=167027]: Uploading segments... Sep 6 11:58:24.744596608 (189h50m21.424792187s) [upload=167027]: Got next segment. Inline: false Sep 6 11:58:24.744640591 (189h50m21.42483616s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 11:58:24.744647484 (189h50m21.424843063s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 11:58:25.259375533 (189h50m21.939571102s) [seg_pos={0 0}, upload=167027]: Begin upload segment... Sep 6 11:58:25.259448612 (189h50m21.939644212s) [seg_pos={0 0}, upload=167027]: Done begin upload segment. Sep 6 11:58:25.259488109 (189h50m21.939683678s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.259552882 (189h50m21.939748451s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260018172 (189h50m21.940213741s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260089489 (189h50m21.940285058s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260352692 (189h50m21.940548251s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260369174 (189h50m21.940564743s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260634693 (189h50m21.940830252s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260733221 (189h50m21.94092878s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.260947791 (189h50m21.94114336s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261026692 (189h50m21.941222261s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261196357 (189h50m21.941391926s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261290317 (189h50m21.941485886s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261437839 (189h50m21.941633408s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261522491 (189h50m21.94171806s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261656628 (189h50m21.941852187s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:25.261737663 (189h50m21.941933232s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Uploading piece... Sep 6 11:58:27.099303196 (189h50m23.779498765s) [upload=167027]: Next returned nil segment Sep 6 11:58:27.099324687 (189h50m23.779520256s) [upload=167027]: Waiting for error group managing segments... Sep 6 11:58:27.393424026 (189h50m24.073619595s) [noise=true , address=45.91.116.14:4190 , piece_id=6G4QGQFS6ZWZW4X67BF3IVYYIY3QNLWGHQSTZC6J36JG6YWLRS5Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.409387483 (189h50m24.089583052s) [noise=true , address=45.91.116.6:4973 , piece_id=JTNML7BJ72XNWHXHLLPHTDJW5YIWJNXR3ZLXQVPUSY5PU7OQXOFA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.429954768 (189h50m24.110150337s) [noise=true , address=45.91.116.8:4047 , piece_id=VLD5RVXWJITM5ZJUW7PJEDLY7KBPYNATNNNBPA3OY2XZLE3BY3DA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.432820041 (189h50m24.11301561s) [noise=true , address=45.91.116.10:4083 , piece_id=TOL2NUHOXXL6ZRP3XHA7EXPEX2W4TT7DW7733O4GHMZ25QL55W2A, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.444927078 (189h50m24.125122647s) [noise=true , address=45.91.116.9:4261 , piece_id=WADCGIXYHRKS4VNTFVFYNJLNWK2XGVMCJLPXWKM4XWCPQUDZBIJQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.446188722 (189h50m24.126384291s) [noise=true , address=45.91.116.10:4082 , piece_id=CM67PH7AA72YSJYTHCYMQJ3ASKWRJAI5MZLBHS5C3ROJ7P4Y6Q3Q, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.461910046 (189h50m24.142105625s) [noise=true , address=45.91.116.14:4189 , piece_id=4O23BEKBHJAM5VWWAGSMN46SOIPABV22MNAVFUI3SDXVBZMX4GEA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.480452909 (189h50m24.160648488s) [noise=true , address=45.91.116.6:4974 , piece_id=HNN3QXD6LGG7Y2R6KHPG3VLGGCIIHJBG5WF5BCX7UA3NTLPWBTVA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.482395105 (189h50m24.162590674s) [noise=true , address=45.91.116.13:4153 , piece_id=54PEBTUPTGTVS6KI43HTKEU4OSTZGDRJCT32UDOGUJE37CPGELBQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.539749154 (189h50m24.219944723s) [noise=true , address=77.90.31.17:4770 , piece_id=3AJVZFSURP3C3ZMCKVCH6NLXDZKDOKIY3D5XBRFD334EXCKZ4FZA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.540488398 (189h50m24.220683967s) [noise=true , address=45.91.116.11:4117 , piece_id=FNAX37RCSRU4VXMKJXO444CXNVSCHCN43M47OMSZNCVDHVATIXJA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.542169786 (189h50m24.222365385s) [noise=true , address=45.91.116.10:4081 , piece_id=FWAXSRLODLBW2Q4DJ7KBFGP2DD7222ZOHUPO4OTKDBXROZTW7QXA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596695102 (189h50m24.276890671s) [noise=true , address=77.90.31.19:4841 , piece_id=CI4ZVCX4PYO6DRN4OG6SECC4CBMLGHQ2ZGOYQ43YLIR6NQKZVWAQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: <nil> Sep 6 11:58:27.596717525 (189h50m24.276913094s) [seg_pos={0 0}, upload=167027]: Segment reached optimal threshold of 13 pieces. Sep 6 11:58:27.596870758 (189h50m24.277066317s) [noise=true , address=45.91.116.12:4226 , piece_id=7IXJJ7OOJJYEYZQ6V56LODJKYFHXZLK4A33YJL34LWVOT5BHITYA, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596949559 (189h50m24.277145128s) [noise=true , address=77.90.31.16:4733 , piece_id=VLIKCTEL6PQRP2FBS3NCTQHBMXS2XBW5CGVDRZEIDRGLSQ5N2LKQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 11:58:27.596986650 (189h50m24.277182219s) [noise=true , address=77.90.31.10:4517 , piece_id=RRVOIOUZ3URPJXUNPTMRGJQANOJVBW2VDO52OW6KY6NN35CIYODQ, seg_pos={0 0}, upload=167027]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->77.90.31.10:4517: use of closed network connection Sep 6 11:58:27.597010826 (189h50m24.277206395s) [seg_pos={0 0}, upload=167027]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 11:58:27.597079277 (189h50m24.277274846s) [upload=167027]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 11:58:27.597083686 (189h50m24.277279245s) [upload=167027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 11:58:27.944227417 (189h50m24.624422986s) [upload=167027]: Done uploading segments... Sep 6 11:58:27.944244640 (189h50m24.624440209s) [upload=167027]: Upload finished. err: <nil> Duration: 6.793617883s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)