Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.509796918s FAIL (2025-10-09 10:00:36 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.509893082s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.275140854s FAIL (2025-10-09 10:00:15 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 6.275202462s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.302423286s FAIL (2025-10-09 09:59:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.302457581s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 7.510217741s FAIL (2025-10-09 09:59:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 7.510298636s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 4.932240437s FAIL (2025-10-09 09:59:27 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.932277879s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 8.823079641s FAIL (2025-10-09 09:58:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 8.823119688s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.09081084s FAIL (2025-10-09 09:58:21 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.090844465s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.033866909s FAIL (2025-10-09 09:57:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.033995863s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 9.953335925s FAIL (2025-10-09 09:57:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 9.953377624s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 4.935060496s FAIL (2025-10-09 09:56:53 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.935094681s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.296557856s FAIL (2025-10-09 09:56:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.296592291s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.922566308s FAIL (2025-10-09 09:56:26 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.92283034s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 13.568436243s FAIL (2025-10-09 09:56:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 13.568477843s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.163380188s FAIL (2025-10-09 09:55:55 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.163409524s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.126879202s FAIL (2025-10-09 09:54:53 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.126917304s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.942216297s FAIL (2025-10-09 09:54:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.942253367s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 4.880584606s FAIL (2025-10-09 09:53:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.880617769s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.334037854s FAIL (2025-10-09 09:53:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.334071116s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.998920714s FAIL (2025-10-09 09:53:24 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.998962744s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.299902654s FAIL (2025-10-09 09:52:55 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.299933382s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 12.675021408s FAIL (2025-10-09 09:52:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 12.675057325s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.088418874s FAIL (2025-10-09 09:52:31 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.088451256s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 13.314708607s FAIL (2025-10-09 09:51:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 13.314739166s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.016792072s FAIL (2025-10-09 09:51:19 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.016861837s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.54857761s FAIL (2025-10-09 09:50:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.548609331s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.010930229s FAIL (2025-10-09 09:50:41 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.010964334s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.357786018s FAIL (2025-10-09 09:50:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.357843246s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.854188892s FAIL (2025-10-09 09:49:53 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.854228006s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.756205563s FAIL (2025-10-09 09:49:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.756236852s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.834344977s FAIL (2025-10-09 09:49:24 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.834378791s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.312689156s FAIL (2025-10-09 09:48:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.312721289s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 4.937694086s FAIL (2025-10-09 09:48:49 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.93772728s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.012079014s FAIL (2025-10-09 09:48:34 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.012109432s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.126348349s FAIL (2025-10-09 09:48:13 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.126382124s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 7.151322371s FAIL (2025-10-09 09:47:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 7.15135356s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.151007155s FAIL (2025-10-09 09:47:45 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.151045366s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 20.114677297s FAIL (2025-10-09 09:46:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 20.114706793s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.048269349s FAIL (2025-10-09 09:46:49 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.048303374s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.038249627s FAIL (2025-10-09 09:46:25 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.038281747s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 18.927822221s FAIL (2025-10-09 09:46:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 18.927889249s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.904617226s FAIL (2025-10-09 09:45:46 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.904652764s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 7.100075875s FAIL (2025-10-09 09:45:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 7.100191737s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.31297043s FAIL (2025-10-09 09:45:25 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.313007771s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 16.163678671s FAIL (2025-10-09 09:45:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 16.163729178s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.005278411s FAIL (2025-10-09 09:44:43 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.005353424s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 8.249387289s FAIL (2025-10-09 09:44:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 8.249423588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 22.530092001s FAIL (2025-10-09 09:43:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 22.530129572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.241250387s FAIL (2025-10-09 09:41:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 6.241287908s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 18.472517888s FAIL (2025-10-09 09:40:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 18.472543917s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 8.203911074s FAIL (2025-10-09 09:40:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 09:29:55.994952189 (379h58m41.457044662s) [upload=334303]: Starting upload Oct 9 09:29:55.995050727 (379h58m41.45714323s) [upload=334303]: Done starting upload Oct 9 09:29:55.995163732 (379h58m41.457256215s) [upload=334303]: Uploading segments... Oct 9 09:29:55.995285513 (379h58m41.457377996s) [upload=334303]: Got next segment. Inline: false Oct 9 09:29:55.995319588 (379h58m41.457412071s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 09:29:55.995324217 (379h58m41.4574167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 09:29:56.279238760 (379h58m41.741331253s) [seg_pos={0 0}, upload=334303]: Begin upload segment... Oct 9 09:29:56.279308062 (379h58m41.741400545s) [seg_pos={0 0}, upload=334303]: Done begin upload segment. Oct 9 09:29:56.279339492 (379h58m41.741431975s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279374308 (379h58m41.741466781s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.279896432 (379h58m41.741988915s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.280604270 (379h58m41.742696753s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281084364 (379h58m41.743176847s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.281540863 (379h58m41.743633346s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.282148170 (379h58m41.744240663s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.283347413 (379h58m41.745439897s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284017760 (379h58m41.746110253s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.284722182 (379h58m41.746814665s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.285742605 (379h58m41.747835088s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.290178611 (379h58m41.752271114s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.293915437 (379h58m41.75600793s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.295113619 (379h58m41.757206102s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.300551213 (379h58m41.762643706s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:56.302883985 (379h58m41.764976478s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Uploading piece... Oct 9 09:29:58.043994747 (379h58m43.50608722s) [upload=334303]: Next returned nil segment Oct 9 09:29:58.044018062 (379h58m43.506110535s) [upload=334303]: Waiting for error group managing segments... Oct 9 09:29:58.331755625 (379h58m43.793848108s) [noise=true , address=145.79.227.2:4114 , piece_id=VWXGQOEUVWXJJ2IG2EBJVUX3KF4JEYCM2MDAHIHVC2MYNTBROMJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.358633105 (379h58m43.820725588s) [noise=true , address=145.79.227.12:4767 , piece_id=GOL364LLBFZTO7KVKZGEJB4Q7OGT4CBVJCIJ73D36UJH2D2G2PWA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.363209210 (379h58m43.825301693s) [noise=true , address=145.79.227.13:4802 , piece_id=X35KYR6DP4O3NFTCBWIAOLAGSUN7W2HB7WXWY2ICVI3NVPZAQQUA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.422198737 (379h58m43.88429122s) [noise=true , address=45.91.116.13:4155 , piece_id=KJ63KVFNJ2PTBPGAYFDXGX2CUO2V5DYV6YSBKTAC4PKTQRJCQN6Q, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.432845973 (379h58m43.894938446s) [noise=true , address=45.91.116.8:4048 , piece_id=PVHZW2OU3GDUH2YYUPPKRGXAO3YXLKTPAXAM2XQKXNV6PPEIQNJQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.433550093 (379h58m43.895642576s) [noise=true , address=45.91.116.9:4262 , piece_id=7Y3LBO6XGNJO6T24CJKBJP2W2YDEYFPMNKYRZ2OO6W4JB3TW335A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.440624394 (379h58m43.902716877s) [noise=true , address=77.90.31.13:4628 , piece_id=UNEADEECIOOR2KHCL4XVCT4B6INEM2SPV4W5QPXDTSO6VRIIKHKA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.443410269 (379h58m43.905502742s) [noise=true , address=45.91.116.14:4189 , piece_id=K6YEEFB5Y7XBAXI2HJT4KKLIUIYQBTVVEOSM4BYBDFZ2XDUK4PRQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.481345486 (379h58m43.943437969s) [noise=true , address=77.90.31.14:4661 , piece_id=5CZDUUJEXXV2YT72F4ZLDUMH723GJ6PIDNRYWYU3VO5OVHPOR2JQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.497161463 (379h58m43.959253936s) [noise=true , address=77.90.31.12:4592 , piece_id=SMVHOV6DTQDMBXLXDQHBBXGDKT4FAOVH2E36SHELMEUHYQICQGPA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.501835312 (379h58m43.963927795s) [noise=true , address=45.91.116.10:4083 , piece_id=5L2UB6RHEL6H44OMGI74DOAP63CWMWILKRJEES72DBXX4SH3RRWQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.571343300 (379h58m44.033435773s) [noise=true , address=77.90.31.18:4807 , piece_id=OGR3O7WD3FFOVL2NHJUFXJZPJW7EBJ44DWX2ZFSZTM6JPRLYE4VQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610827687 (379h58m44.07292017s) [noise=true , address=77.90.31.12:4590 , piece_id=VON23P3IJWEPBBCIBK6AE7DYU73UAE6T7OETPMGZE5X3PICLWATQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: <nil> Oct 9 09:29:58.610855530 (379h58m44.072948013s) [seg_pos={0 0}, upload=334303]: Segment reached optimal threshold of 13 pieces. Oct 9 09:29:58.610992511 (379h58m44.073084994s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611003962 (379h58m44.073096445s) [noise=true , address=145.79.227.18:4007 , piece_id=WURYLP5HYUGMOH73WNPOTR4ZEVYRJCBGBLDWLC7KVLG754XCIS4A, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611134972 (379h58m44.073227455s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611143678 (379h58m44.073236151s) [noise=true , address=145.79.227.3:4151 , piece_id=PAAMT43W3D7WH7BLBFCKXNB37KKBHWE2IYKBO43SFV3SUDEZ7MUQ, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611264589 (379h58m44.073357062s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Oct 9 09:29:58.611273115 (379h58m44.073365598s) [noise=true , address=145.79.227.17:4972 , piece_id=R2P3A6QWRU2UCQH6FVD4VITPOJTJJGMYAIB5QMRZQIX7OVVBGLAA, seg_pos={0 0}, upload=334303]: Long tail cancellation due to optimal threshold Oct 9 09:29:58.611292623 (379h58m44.073385106s) [seg_pos={0 0}, upload=334303]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 09:29:58.611379197 (379h58m44.07347167s) [upload=334303]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 09:29:58.611386310 (379h58m44.073478783s) [upload=334303]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 09:29:58.956076653 (379h58m44.418169136s) [upload=334303]: Done uploading segments... Oct 9 09:29:58.956305469 (379h58m44.418397973s) [upload=334303]: Upload finished. err: <nil> Duration: 8.203945128s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)