Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....................X.....XXXXXXXX.XXX.X....................] duration: 6.490704688s FAIL (2025-10-07 07:05:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.490746547s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........................X....X.XXX.XXXX....................] duration: 6.064109006s FAIL (2025-10-07 07:05:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.064146226s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........................X....X.XXX.XXXX...................] duration: 6.068026667s FAIL (2025-10-07 07:05:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.068062003s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....................X.....XXXXXXXX.XXX.X...................] duration: 6.058413321s FAIL (2025-10-07 07:05:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.058452265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...........................X....X.XXX.XXXX..................] duration: 6.01488766s FAIL (2025-10-07 07:04:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.014945751s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [............................X....X.XXX.XXXX.................] duration: 6.142977883s FAIL (2025-10-07 07:04:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.143016026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [......................X.....XXXXXXXX.XXX.X..................] duration: 6.068886632s FAIL (2025-10-07 07:03:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.068915217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................X....X.XXX.XXXX................] duration: 5.877868201s FAIL (2025-10-07 07:03:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.877895232s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......................X.....XXXXXXXX.XXX.X.................] duration: 6.305005912s FAIL (2025-10-07 07:03:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.305039596s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........................X.....XXXXXXXX.XXX.X................] duration: 6.365914794s FAIL (2025-10-07 07:03:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.365965861s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................X....X.XXX.XXXX...............] duration: 6.015112459s FAIL (2025-10-07 07:02:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.015141574s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................X.....XXXXXXXX.XXX.X...............] duration: 6.112432863s FAIL (2025-10-07 07:02:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.112511482s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................X....X.XXX.XXXX..............] duration: 6.138350964s FAIL (2025-10-07 07:02:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.138388666s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........................X.....XXXXXXXX.XXX.X..............] duration: 6.254497677s FAIL (2025-10-07 07:01:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.254561077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................X....X.XXX.XXXX.............] duration: 6.606739154s FAIL (2025-10-07 07:01:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.606793728s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...........................X.....XXXXXXXX.XXX.X.............] duration: 6.110451009s FAIL (2025-10-07 07:00:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.110482869s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................X....X.XXX.XXXX............] duration: 6.137512437s FAIL (2025-10-07 07:00:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.137623759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [............................X.....XXXXXXXX.XXX.X............] duration: 6.561941371s FAIL (2025-10-07 07:00:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.561982188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.............................X.....XXXXXXXX.XXX.X...........] duration: 6.457264699s FAIL (2025-10-07 07:00:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.457303072s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..............................X.....XXXXXXXX.XXX.X..........] duration: 6.136656738s FAIL (2025-10-07 06:59:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.136687196s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................................X....X.XXX.XXXX...........] duration: 5.950645961s FAIL (2025-10-07 06:59:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.950679626s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................X.....XXXXXXXX.XXX.X.........] duration: 6.288531212s FAIL (2025-10-07 06:58:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.288559557s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................X....X.XXX.XXXX..........] duration: 5.912454672s FAIL (2025-10-07 06:58:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.912500211s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [....................................X....X.XXX.XXXX.........] duration: 5.951404527s FAIL (2025-10-07 06:57:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.95143194s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [................................X.....XXXXXXXX.XXX.X........] duration: 6.363082901s FAIL (2025-10-07 06:57:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.363113571s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....................................X....X.XXX.XXXX........] duration: 6.056124781s FAIL (2025-10-07 06:57:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.05616628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.................................X.....XXXXXXXX.XXX.X.......] duration: 6.450142319s FAIL (2025-10-07 06:56:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.450178477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [......................................X....X.XXX.XXXX.......] duration: 6.07518784s FAIL (2025-10-07 06:56:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.075223308s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..................................X.....XXXXXXXX.XXX.X......] duration: 6.198493327s FAIL (2025-10-07 06:56:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.198528285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.......................................X....X.XXX.XXXX......] duration: 6.118428161s FAIL (2025-10-07 06:55:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.118477264s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [........................................X....X.XXX.XXXX.....] duration: 5.845763712s FAIL (2025-10-07 06:55:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.845893499s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...................................X.....XXXXXXXX.XXX.X.....] duration: 6.240946605s FAIL (2025-10-07 06:54:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.240975079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........................................X....X.XXX.XXXX....] duration: 5.995501506s FAIL (2025-10-07 06:54:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.995559236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....................................X.....XXXXXXXX.XXX.X....] duration: 6.426677444s FAIL (2025-10-07 06:54:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.426710718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........................................X....X.XXX.XXXX...] duration: 6.001718302s FAIL (2025-10-07 06:54:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.001751415s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....................................X.....XXXXXXXX.XXX.X...] duration: 6.25547893s FAIL (2025-10-07 06:54:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.255517173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...........................................X....X.XXX.XXXX..] duration: 6.073463308s FAIL (2025-10-07 06:53:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 6.073488777s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [......................................X.....XXXXXXXX.XXX.X..] duration: 7.762113755s FAIL (2025-10-07 06:52:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 7.762145036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [............................................X....X.XXX.XXXX.] duration: 5.984059398s FAIL (2025-10-07 06:52:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 5.984090297s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................................X....X.XXX.XXXX] duration: 1m0.003867553s FAIL (2025-10-07 06:52:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0m13.663786069s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.201972678 (329h20m13.664065151s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202214568 (329h20m13.664307041s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202477639 (329h20m13.664570112s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202719019 (329h20m13.664811492s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.202954858 (329h20m13.665047341s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203195988 (329h20m13.665288471s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203444370 (329h20m13.665536843s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.203709945 (329h20m13.665802429s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204029956 (329h20m13.666122429s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204293427 (329h20m13.6663859s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204564904 (329h20m13.666657387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.204897538 (329h20m13.666990011s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205219200 (329h20m13.667311683s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:28.205527277 (329h20m13.66761975s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Uploading piece... Oct 7 06:51:29.736522626 (329h20m15.198615099s) [upload=289826]: Next returned nil segment Oct 7 06:51:29.736541020 (329h20m15.198633503s) [upload=289826]: Waiting for error group managing segments... Oct 7 06:51:30.125616866 (329h20m15.587709349s) [noise=true , address=77.90.31.13:4645 , piece_id=SXWFSXCXGCJ6JQQ3TLUSXISLZOZPZ5F7G27YMMWJR2HS4LIACQRQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.137671500 (329h20m15.599763983s) [noise=true , address=45.91.116.5:4314 , piece_id=ELMJ3WLDCD3O3DU34HQBXHZK363PRFIEUOHDRR3MF5R23HP2KIVQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.171741454 (329h20m15.633833927s) [noise=true , address=45.91.116.4:4942 , piece_id=T63PJRNBHMZBHRCNVAXIP2MVYADHNY7CWYMY5R5PWTOBOORRTZEA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.193322979 (329h20m15.655415462s) [noise=true , address=45.91.116.4:4971 , piece_id=3DLHYCS4KUFQHCCTIOFTMKABKV7H5LB5K3UFSBJ7I4AAQBS5ONSQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.209701726 (329h20m15.67179421s) [noise=true , address=45.91.116.7:4020 , piece_id=PZUL75P6KSD4L7JAVLFK2LDCX6GHDJZ763XIN256QHG5T7UA7S6Q, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.218096993 (329h20m15.680189476s) [noise=true , address=77.90.31.10:4542 , piece_id=UTXGCBORDPRLX3KMCA6MQ5JQZOAYHBZ67MVZM7MRKC3745KRJPTQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.249328143 (329h20m15.711420636s) [noise=true , address=45.91.116.6:4993 , piece_id=SWWNGUZTY2VVLZ4SRX4TG62HN3ASKZIAO4ITM6ZIZBKAG24YBVZQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.300300332 (329h20m15.762392815s) [noise=true , address=77.90.31.15:4732 , piece_id=N3K6T6SBE4QULSC3NNQAP2IGPKMY3F3TPDF2WHB5O6T2JDRAQKZA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:30.612615722 (329h20m16.074708205s) [noise=true , address=77.90.31.18:4823 , piece_id=L6RW4RMRJJNIDEGVH4EXCUHTGWM4EBZG4K4DQAJCQPFKADWZGGIA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:51:31.000131546 (329h20m16.462224039s) [noise=true , address=77.90.31.14:4681 , piece_id=PI3OENRLUYB3P2CXJ3UDBV3DSPFWKIUIMEJS5VKWNJQWLHP2M37A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:21.297772550 (329h21m6.759865033s) [noise=true , address=145.79.227.23:4565 , piece_id=GDGUAZTT3AAZTIW67VTX6GWBS7ANAQ3RQ6KNAXYHY4RBB7HYACNQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.477208506 (329h21m8.939300989s) [noise=true , address=145.79.227.28:4746 , piece_id=TKE37ED7JSCLOSTTXBQLT4W4GXYIRYOTUBUE2LHVZKCHIU7WBDAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.656318130 (329h21m9.118410613s) [noise=true , address=145.79.227.30:4788 , piece_id=CJTC7BNPJE5EWNFHGTDWTIYW46LO5FYG5NZFESHGL2X4U4GYIUPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817377904 (329h21m9.279470387s) [noise=true , address=145.79.227.33:4897 , piece_id=MJMUATPYYOLIGIFQN6WATJGLROHKKN6WJAKDEO5QDDZ5ZRX3PWPA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: <nil> Oct 7 06:52:23.817425264 (329h21m9.279517777s) [seg_pos={0 0}, upload=289826]: Segment reached optimal threshold of 14 pieces. Oct 7 06:52:23.817666694 (329h21m9.279759187s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59964->145.79.227.32:4841: use of closed network connection Oct 7 06:52:23.817688254 (329h21m9.279780737s) [noise=true , address=145.79.227.32:4841 , piece_id=E6XSNV5BTKLRLRF3RGN2TX2Z33644MM46WDUFKSCAXG25CTPNS3A, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817768908 (329h21m9.279861391s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Y936LLJepKpXAF6huSDRd61pWHNmdntRVNZutbadfeaZerMyj): protocol: read tcp 188.166.225.170:38192->145.79.227.44:4200: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.817780440 (329h21m9.279872923s) [noise=true , address=145.79.227.44:4200 , piece_id=C5DTAOCSU37PPYECFP7DGZMJC7AJRB7XHDNGA6QQ5IZSWOA44WAQ, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817898866 (329h21m9.279991339s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh): protocol: context canceled; context canceled Oct 7 06:52:23.817907172 (329h21m9.279999645s) [noise=true , address=145.79.227.31:4820 , piece_id=UHW6SB4UM4ZNTVOMNOTWEAQDZ5WJWL3NTWNGEB4KOHQNSOCTTJVA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.817998645 (329h21m9.280091118s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: read tcp 188.166.225.170:49366->145.79.227.49:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:52:23.818007192 (329h21m9.280099675s) [noise=true , address=145.79.227.49:4356 , piece_id=WOSFNISNO6JBO4NFK43I666MZ3NVLFG3EWZPEWU7UUD57UXAVISA, seg_pos={0 0}, upload=289826]: Long tail cancellation due to optimal threshold Oct 7 06:52:23.818027320 (329h21m9.280119803s) [seg_pos={0 0}, upload=289826]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:52:23.818099567 (329h21m9.280192051s) [upload=289826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:52:23.818104236 (329h21m9.280196709s) [upload=289826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:52:24.148601679 (329h21m9.610694162s) [upload=289826]: Done uploading segments... Oct 7 06:52:24.148623171 (329h21m9.610715644s) [upload=289826]: Upload finished. err: <nil> Duration: 1m0.003884335s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......................................X.....XXXXXXXX.XXX.X.] duration: 6.523136199s FAIL (2025-10-07 06:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 6.523155905s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........................................X.....XXXXXXXX.XXX.X] duration: 30.002602686s FAIL (2025-10-07 06:52:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ) piecestore close: write tcp 188.166.225.170:43094->77.90.31.17:4770: use of closed network connection; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA) context deadline exceeded ======LOGS====== ad=289831]: Starting upload Oct 7 06:51:50.235180289 (329h20m35.697272762s) [upload=289831]: Done starting upload Oct 7 06:51:50.235360752 (329h20m35.697453235s) [upload=289831]: Uploading segments... Oct 7 06:51:50.235478376 (329h20m35.697570859s) [upload=289831]: Got next segment. Inline: false Oct 7 06:51:50.235526288 (329h20m35.697618761s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:51:50.235530836 (329h20m35.697623319s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:51:50.489386760 (329h20m35.951479243s) [seg_pos={0 0}, upload=289831]: Begin upload segment... Oct 7 06:51:50.489446053 (329h20m35.951538537s) [seg_pos={0 0}, upload=289831]: Done begin upload segment. Oct 7 06:51:50.489472454 (329h20m35.951564927s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489483775 (329h20m35.951576248s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.489946706 (329h20m35.952039179s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490539956 (329h20m35.952632429s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.490654204 (329h20m35.952746687s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491086637 (329h20m35.95317911s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491158584 (329h20m35.953251077s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491426685 (329h20m35.953519158s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491618901 (329h20m35.953711384s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491733990 (329h20m35.953826473s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.491983616 (329h20m35.954076089s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492051896 (329h20m35.954144379s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492318544 (329h20m35.954411027s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492390821 (329h20m35.954483294s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.492812815 (329h20m35.954905298s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:50.493177759 (329h20m35.955270242s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Uploading piece... Oct 7 06:51:52.173664983 (329h20m37.635757466s) [upload=289831]: Next returned nil segment Oct 7 06:51:52.173686854 (329h20m37.635779337s) [upload=289831]: Waiting for error group managing segments... Oct 7 06:51:52.559486130 (329h20m38.021578613s) [noise=true , address=45.91.116.7:4011 , piece_id=QU6O55VDVCJQPU75OQ7ZWATTMRXV6PN4UWLHK5ALZPIAMCV6KKNA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.563137593 (329h20m38.025230076s) [noise=true , address=77.90.31.21:4877 , piece_id=U3JLFNUJ43KZY6U7KEY6XDG6AIIWTURWBJ3TFT7UIRV72JBEFX3A, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.568473153 (329h20m38.030565636s) [noise=true , address=45.91.116.13:4155 , piece_id=HHE5DWD357RSDPTODHWC6BFR3TT66OW3GPLODXN35CT7W67I6TGA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.571556784 (329h20m38.033649268s) [noise=true , address=77.90.31.17:4770 , piece_id=E3LUCR27SIFLAIPKRLC7KY553VRFNYBVWAFLYCLMASO3MYKZ6MPQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.610893538 (329h20m38.072986011s) [noise=true , address=45.91.116.12:4225 , piece_id=AGJWX4QX2VA2TZP36TYRH3ZA2Y6T4XSH5SIHM4IB4MMZQFFTHJMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.637723106 (329h20m38.099815599s) [noise=true , address=77.90.31.13:4627 , piece_id=XPICPS43T7OSSL4YUCSWUR255K2NRW2JS3VUR5LZBJ5TZ6NF3LMA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.638664629 (329h20m38.100757112s) [noise=true , address=77.90.31.16:4734 , piece_id=N2R6ZMYWWYKKBYVCANPTOG2P2KGLIYCMDGKFDJZFA33STMSFT6GQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.671457492 (329h20m38.133549975s) [noise=true , address=45.91.116.13:4154 , piece_id=4CTWENWVA33KNZZVX6D2EIHCEXW7NDFEK6WJURYVROTUB2PY4UHQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.683291536 (329h20m38.145384019s) [noise=true , address=45.91.116.14:4189 , piece_id=BDOAKV2BCAFX6YJGKO5WVSQBEMIPW2PFI7J4DSJZU7JMXRGR3NOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.693680980 (329h20m38.155773463s) [noise=true , address=77.90.31.10:4519 , piece_id=T2HJJKNIRP3XRG4OF5P5RERIFYL4KQXL25LT76WV7XZQ4CO72BOA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.730622553 (329h20m38.192715036s) [noise=true , address=45.91.116.5:4299 , piece_id=IMTJ5NPLVY6FDHRPIILEBNYEU4CVCXBW2VEEXICIZS5FAE6XTEYA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.734749601 (329h20m38.196842084s) [noise=true , address=45.91.116.5:4297 , piece_id=2WP5Z4RSLVO553UEWNYVR5EUMNI35LNQUFSZUETILZXEMMKDCA3Q, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784458485 (329h20m38.246550968s) [noise=true , address=45.91.116.11:4117 , piece_id=BJ3VSI3BGZ2DVCMM4G4HSDJRT3LKPIMHUOT62WNI3HLMQIB3R6MA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: <nil> Oct 7 06:51:52.784487500 (329h20m38.246579984s) [seg_pos={0 0}, upload=289831]: Segment reached optimal threshold of 13 pieces. Oct 7 06:51:52.785353379 (329h20m38.247445872s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: rpc: dial tcp 145.79.227.2:4115: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785369680 (329h20m38.247462163s) [noise=true , address=145.79.227.2:4115 , piece_id=QNLWYEEK6KKEZ5F7HHMHJJMS6CFOVEQLVOV3WQF63WFBDCLLA7UA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785579379 (329h20m38.247671862s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: read tcp 188.166.225.170:47296->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785591773 (329h20m38.247684257s) [noise=true , address=145.79.227.16:4910 , piece_id=L4SVDIJX27ZY4IL6JZ2PHWMLBMJA7YGLGC6J5BYSAT64WF4UDYTA, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785726520 (329h20m38.247819013s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: read tcp 188.166.225.170:36690->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:52.785735808 (329h20m38.247828291s) [noise=true , address=145.79.227.19:4078 , piece_id=TYNHZIMXTHXGBIWEMUPCRD2C2EXHOPBLCV2CUDV4OHSIDFTOQXZQ, seg_pos={0 0}, upload=289831]: Long tail cancellation due to optimal threshold Oct 7 06:51:52.785754523 (329h20m38.247847006s) [seg_pos={0 0}, upload=289831]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:51:52.786067308 (329h20m38.248159781s) [upload=289831]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:51:52.786077008 (329h20m38.248169491s) [upload=289831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:51:53.221867805 (329h20m38.683960288s) [upload=289831]: Done uploading segments... Oct 7 06:51:53.221893353 (329h20m38.683985837s) [upload=289831]: Upload finished. err: <nil> Duration: 30.002631932s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................................X....X.XXX.XXX] duration: 1m0.248768624s FAIL (2025-10-07 06:51:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== :27.700102899 (329h19m13.162195382s) [seg_pos={0 0}, upload=289811]: Begin upload segment... Oct 7 06:50:27.700178302 (329h19m13.162270785s) [seg_pos={0 0}, upload=289811]: Done begin upload segment. Oct 7 06:50:27.700215363 (329h19m13.162307836s) [noise=true , address=64.49.13.170:4727 , piece_id=DONA6PSAIPHETDJOKQAKT4DMQRYIBFOYMOTWINY4NXSTH6COZ4HQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.700228387 (329h19m13.16232087s) [noise=true , address=77.90.31.15:4713 , piece_id=ALGG3ODBJPDDV352S35X7DF5BAFQJLMMDGF5MJM264HCFM7Q22TA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.700285566 (329h19m13.162378049s) [noise=true , address=45.91.116.8:4051 , piece_id=IMDOYM3BJQU47QTW4JSFUHBH3PCRZQWY4KGKT32RID2MWREBQTMA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.700912991 (329h19m13.163005474s) [noise=true , address=77.90.31.17:4775 , piece_id=OCIMDXJFAJL4FSYINEPVTUJGIIWOQUUES5JA3OZZS4ORUD3ZSXJA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.701408144 (329h19m13.163500627s) [noise=true , address=145.79.227.25:4624 , piece_id=Y52EAB4ACN4NMWTWMAVKIA62TQP6Q26P3SDZWTVMHZX34PKVNOVQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.701849585 (329h19m13.163942078s) [noise=true , address=77.90.31.19:4847 , piece_id=KJ2JYOS5SK27IYSHAVCEGP2IXMQJEG2LXBCHFLWK5FXBGWMGPLNQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.701925339 (329h19m13.164017822s) [noise=true , address=145.79.227.43:4190 , piece_id=OQLIJEBVWH5GWIW7BAQYA4BHPEUN6TWNSXA6KPWMPASLKH4TT5IA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.702264434 (329h19m13.164356917s) [noise=true , address=45.91.116.10:4106 , piece_id=23AYAPTHQRJOOUSVSOBD46F4G2PMZA3OUNN7ZG467FC2L7DOH7HA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.702456881 (329h19m13.164549384s) [noise=true , address=45.91.116.13:4165 , piece_id=ZRBIOBMLGSZZHKVCVO5V2YTAC7XPHPTKOFE3W5SW2KLKUPUR5TJA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.702640310 (329h19m13.164732793s) [noise=true , address=145.79.227.52:4467 , piece_id=6MMMGGN2JFKA2SJORFXVKOEWI525LYP4L4NNUCWXWZFYGX7RPXNQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.703049158 (329h19m13.165141651s) [noise=true , address=145.79.227.32:4844 , piece_id=2HEVFK2O3NSQNMZCBRBTDFHDAAYEU6TWZP2E7BAFO53ZFNWXIKRA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.703448969 (329h19m13.165541452s) [noise=true , address=77.90.31.11:4580 , piece_id=T75CO3VYEYMGBMBTVRDIG7P6XBLFUA5L55PWBVNF3F7Y7GRIZ72A, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.703732881 (329h19m13.165825364s) [noise=true , address=45.91.116.14:4207 , piece_id=FPGCODOSCF2WBATCGEDSDTAV4K44MJ62DYAC2IC3ZULA4MD3DTDQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.703852368 (329h19m13.165944851s) [noise=true , address=45.91.116.4:4957 , piece_id=HNTP2GIE3RDCX3VKKMC74DTYM64H4C4DJ6XNK35EF4UMTYGYHNTQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.704380784 (329h19m13.166473267s) [noise=true , address=77.90.31.10:4544 , piece_id=GAUTS7JWMTID4OB55GGETK2UBUWMO7XIAVOJRZD7I2RF5TIMGLFA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.704843916 (329h19m13.166936399s) [noise=true , address=45.91.116.5:4326 , piece_id=E4C4YJLCAVKV3UNCL7ARQFF3PSYOELWTZCLVD6VDJWHYQBFRNUHQ, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.705337426 (329h19m13.167429909s) [noise=true , address=145.79.227.44:4212 , piece_id=3CQDYU2T7WRJD5SA7EODB23P2HTPSO4VDSED42AZHH5NUJDZBKOA, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:27.706054140 (329h19m13.168146653s) [noise=true , address=145.79.227.22:4517 , piece_id=RC3ZO3SU5GVAOMXYGKYHY4YNN77RSLWGODFKWANICHIBPPM2DL5Q, seg_pos={0 0}, upload=289811]: Uploading piece... Oct 7 06:50:29.255089088 (329h19m14.717181571s) [upload=289811]: Next returned nil segment Oct 7 06:50:29.255115990 (329h19m14.717208473s) [upload=289811]: Waiting for error group managing segments... Oct 7 06:50:29.648312350 (329h19m15.110404833s) [noise=true , address=77.90.31.15:4713 , piece_id=ALGG3ODBJPDDV352S35X7DF5BAFQJLMMDGF5MJM264HCFM7Q22TA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.671037252 (329h19m15.133129736s) [noise=true , address=45.91.116.10:4106 , piece_id=23AYAPTHQRJOOUSVSOBD46F4G2PMZA3OUNN7ZG467FC2L7DOH7HA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.675966769 (329h19m15.138059252s) [noise=true , address=45.91.116.5:4326 , piece_id=E4C4YJLCAVKV3UNCL7ARQFF3PSYOELWTZCLVD6VDJWHYQBFRNUHQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.679362395 (329h19m15.141454878s) [noise=true , address=77.90.31.19:4847 , piece_id=KJ2JYOS5SK27IYSHAVCEGP2IXMQJEG2LXBCHFLWK5FXBGWMGPLNQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.706026207 (329h19m15.16811869s) [noise=true , address=45.91.116.13:4165 , piece_id=ZRBIOBMLGSZZHKVCVO5V2YTAC7XPHPTKOFE3W5SW2KLKUPUR5TJA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.788974565 (329h19m15.251067048s) [noise=true , address=77.90.31.11:4580 , piece_id=T75CO3VYEYMGBMBTVRDIG7P6XBLFUA5L55PWBVNF3F7Y7GRIZ72A, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.800870486 (329h19m15.262962969s) [noise=true , address=45.91.116.8:4051 , piece_id=IMDOYM3BJQU47QTW4JSFUHBH3PCRZQWY4KGKT32RID2MWREBQTMA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:29.928370477 (329h19m15.39046297s) [noise=true , address=45.91.116.14:4207 , piece_id=FPGCODOSCF2WBATCGEDSDTAV4K44MJ62DYAC2IC3ZULA4MD3DTDQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:30.399978727 (329h19m15.86207121s) [noise=true , address=77.90.31.17:4775 , piece_id=OCIMDXJFAJL4FSYINEPVTUJGIIWOQUUES5JA3OZZS4ORUD3ZSXJA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:31.293400117 (329h19m16.75549262s) [noise=true , address=45.91.116.4:4957 , piece_id=HNTP2GIE3RDCX3VKKMC74DTYM64H4C4DJ6XNK35EF4UMTYGYHNTQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:31.382595460 (329h19m16.844687943s) [noise=true , address=77.90.31.10:4544 , piece_id=GAUTS7JWMTID4OB55GGETK2UBUWMO7XIAVOJRZD7I2RF5TIMGLFA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:50:37.625871751 (329h19m23.087964234s) [noise=true , address=64.49.13.170:4727 , piece_id=DONA6PSAIPHETDJOKQAKT4DMQRYIBFOYMOTWINY4NXSTH6COZ4HQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: <nil> Oct 7 06:51:26.703685633 (329h20m12.165778116s) [noise=true , address=145.79.227.32:4844 , piece_id=2HEVFK2O3NSQNMZCBRBTDFHDAAYEU6TWZP2E7BAFO53ZFNWXIKRA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR, address:145.79.227.32:4844): protocol: rpc: dial tcp 145.79.227.32:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.703841660 (329h20m12.165934143s) [noise=true , address=145.79.227.44:4212 , piece_id=3CQDYU2T7WRJD5SA7EODB23P2HTPSO4VDSED42AZHH5NUJDZBKOA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:145.79.227.44:4212): protocol: rpc: dial tcp 145.79.227.44:4212: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.703978901 (329h20m12.166071384s) [noise=true , address=145.79.227.43:4190 , piece_id=OQLIJEBVWH5GWIW7BAQYA4BHPEUN6TWNSXA6KPWMPASLKH4TT5IA, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:1eKHcxWuBbQAGX44kVrn9cfEhEcRpmQfQegcoMGjr92pfM1VVF, address:145.79.227.43:4190): protocol: read tcp 188.166.225.170:58180->145.79.227.43:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.704029166 (329h20m12.166121649s) [noise=true , address=145.79.227.25:4624 , piece_id=Y52EAB4ACN4NMWTWMAVKIA62TQP6Q26P3SDZWTVMHZX34PKVNOVQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:145.79.227.25:4624): protocol: read tcp 188.166.225.170:34440->145.79.227.25:4624: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.704098158 (329h20m12.166190731s) [noise=true , address=145.79.227.22:4517 , piece_id=RC3ZO3SU5GVAOMXYGKYHY4YNN77RSLWGODFKWANICHIBPPM2DL5Q, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:1wSJWmZaz4FNEiar3UGW17tJc6XvaW4erEXbpAsWnE6FvuLRzT, address:145.79.227.22:4517): protocol: read tcp 188.166.225.170:55492->145.79.227.22:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.704117074 (329h20m12.166209557s) [noise=true , address=145.79.227.52:4467 , piece_id=6MMMGGN2JFKA2SJORFXVKOEWI525LYP4L4NNUCWXWZFYGX7RPXNQ, seg_pos={0 0}, upload=289811]: Done uploading piece. err: ecclient: upload failed (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT, address:145.79.227.52:4467): protocol: read tcp 188.166.225.170:42916->145.79.227.52:4467: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:51:26.704143805 (329h20m12.166236288s) [seg_pos={0 0}, upload=289811]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Oct 7 06:51:26.950837070 (329h20m12.412929553s) [upload=289811]: Done uploading segments... Oct 7 06:51:26.950871706 (329h20m12.412964179s) [upload=289811]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.248784624s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........................................X.....XXXXXXXX.XXX.] duration: 26.365443873s FAIL (2025-10-07 06:51:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, Piece ID: B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A) context deadline exceeded ======LOGS====== load=289802]: Starting upload Oct 7 06:49:43.013416725 (329h18m28.475509208s) [upload=289802]: Done starting upload Oct 7 06:49:43.013521785 (329h18m28.475614258s) [upload=289802]: Uploading segments... Oct 7 06:49:43.013671580 (329h18m28.475764063s) [upload=289802]: Got next segment. Inline: false Oct 7 06:49:43.013748838 (329h18m28.475841311s) [upload=289802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:49:43.013756813 (329h18m28.475849316s) [upload=289802]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:49:43.301776509 (329h18m28.763868992s) [seg_pos={0 0}, upload=289802]: Begin upload segment... Oct 7 06:49:43.301892891 (329h18m28.763985374s) [seg_pos={0 0}, upload=289802]: Done begin upload segment. Oct 7 06:49:43.301914141 (329h18m28.764006614s) [noise=true , address=45.91.116.9:4262 , piece_id=B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.301925783 (329h18m28.764018256s) [noise=true , address=77.90.31.10:4517 , piece_id=I3IXWV4LB57CPF7DWZR5WHDE4WCLTEWPMLVLENYK7PJCKOPUNL4A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302003843 (329h18m28.764096326s) [noise=true , address=77.90.31.18:4805 , piece_id=7GXSRX6YF3NXI7KKDZNEXW55O564OJPCMMLZS5NKV5FPZJYOGNWA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302532549 (329h18m28.764625032s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302987566 (329h18m28.765080049s) [noise=true , address=45.91.116.13:4155 , piece_id=B3TIGHP4NK4OI442VHX4HKJN3ZALOGMAVPHS5MB44TXTULMWUW6A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.303281294 (329h18m28.765373777s) [noise=true , address=45.91.116.5:4298 , piece_id=RMKPPLJGNJ3H5QBOXWVBMRJ6ECFEZCAT77XGWEQF7ZCNYMK7RNQQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.303691746 (329h18m28.765784229s) [noise=true , address=77.90.31.11:4555 , piece_id=4TNNCKUFE7OJIFXDTQUXZB4U6AKL2TWWS32YTJVINFFO7AOBTE2Q, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.305942041 (329h18m28.768034535s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.306481458 (329h18m28.768573941s) [noise=true , address=45.91.116.4:4937 , piece_id=M3NYCLHYZH6MNIEH3UWWAQWX7XWAWQX6CIXBFOSXS46V6MIHHCMA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.306995507 (329h18m28.76908799s) [noise=true , address=45.91.116.11:4117 , piece_id=JCZ23FW2MQKQ6G6UPLTLMDSATKPNXPNTFPXNXAHZC4RWDHCIZ6PA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.308596987 (329h18m28.77068947s) [noise=true , address=45.91.116.6:4975 , piece_id=XFBC2BPMDTSUWBXIDUBYSTCEQ4D4DXZKXNMF5BQ446FPAIYMHLSA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.310241648 (329h18m28.772334121s) [noise=true , address=77.90.31.10:4518 , piece_id=AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.311645512 (329h18m28.773737995s) [noise=true , address=77.90.31.14:4661 , piece_id=GCFDREABYIPQMFULQLL25B2UEANXVGXVWUJAB3S6M4OCRNGCR27A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.313248414 (329h18m28.775340897s) [noise=true , address=45.91.116.4:4940 , piece_id=5RYKMI6YCCYT3KLZFONVARPDRRIG6DBSJA6UQUVSVK4EBQJ6FHNA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.314782415 (329h18m28.776874888s) [noise=true , address=45.91.116.13:4154 , piece_id=4JKN275HNOUS67UHI6DFS7RYHDEKG5FQD4NJ4CY4XHCDENP65HYA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.314970062 (329h18m28.777062545s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:45.075789771 (329h18m30.537882284s) [upload=289802]: Next returned nil segment Oct 7 06:49:45.075816292 (329h18m30.537908795s) [upload=289802]: Waiting for error group managing segments... Oct 7 06:49:45.376070605 (329h18m30.838163088s) [noise=true , address=77.90.31.11:4555 , piece_id=4TNNCKUFE7OJIFXDTQUXZB4U6AKL2TWWS32YTJVINFFO7AOBTE2Q, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.416112341 (329h18m30.878204814s) [noise=true , address=45.91.116.5:4298 , piece_id=RMKPPLJGNJ3H5QBOXWVBMRJ6ECFEZCAT77XGWEQF7ZCNYMK7RNQQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.427448547 (329h18m30.88954102s) [noise=true , address=45.91.116.4:4937 , piece_id=M3NYCLHYZH6MNIEH3UWWAQWX7XWAWQX6CIXBFOSXS46V6MIHHCMA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.448475256 (329h18m30.910567729s) [noise=true , address=45.91.116.13:4154 , piece_id=4JKN275HNOUS67UHI6DFS7RYHDEKG5FQD4NJ4CY4XHCDENP65HYA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.482005883 (329h18m30.944098357s) [noise=true , address=45.91.116.6:4975 , piece_id=XFBC2BPMDTSUWBXIDUBYSTCEQ4D4DXZKXNMF5BQ446FPAIYMHLSA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.482247453 (329h18m30.944339936s) [noise=true , address=77.90.31.14:4661 , piece_id=GCFDREABYIPQMFULQLL25B2UEANXVGXVWUJAB3S6M4OCRNGCR27A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.490005565 (329h18m30.952098048s) [noise=true , address=77.90.31.10:4518 , piece_id=AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.496317425 (329h18m30.958409908s) [noise=true , address=45.91.116.4:4940 , piece_id=5RYKMI6YCCYT3KLZFONVARPDRRIG6DBSJA6UQUVSVK4EBQJ6FHNA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.500978250 (329h18m30.963070733s) [noise=true , address=77.90.31.18:4805 , piece_id=7GXSRX6YF3NXI7KKDZNEXW55O564OJPCMMLZS5NKV5FPZJYOGNWA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.515497326 (329h18m30.977589799s) [noise=true , address=45.91.116.13:4155 , piece_id=B3TIGHP4NK4OI442VHX4HKJN3ZALOGMAVPHS5MB44TXTULMWUW6A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.849733706 (329h18m31.311826179s) [noise=true , address=45.91.116.9:4262 , piece_id=B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:46.032133195 (329h18m31.494225669s) [noise=true , address=77.90.31.10:4517 , piece_id=I3IXWV4LB57CPF7DWZR5WHDE4WCLTEWPMLVLENYK7PJCKOPUNL4A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:51.351895502 (329h18m36.813987975s) [noise=true , address=45.91.116.11:4117 , piece_id=JCZ23FW2MQKQ6G6UPLTLMDSATKPNXPNTFPXNXAHZC4RWDHCIZ6PA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:51.351933755 (329h18m36.814026238s) [seg_pos={0 0}, upload=289802]: Segment reached optimal threshold of 13 pieces. Oct 7 06:49:51.352216383 (329h18m36.814308856s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 145.79.227.14:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352226461 (329h18m36.814318935s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352248384 (329h18m36.814340867s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: read tcp 188.166.225.170:56122->145.79.227.5:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352257280 (329h18m36.814349763s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352361849 (329h18m36.814454353s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: read tcp 188.166.225.170:57450->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352371328 (329h18m36.814463821s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352391606 (329h18m36.814484079s) [seg_pos={0 0}, upload=289802]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:49:51.352469234 (329h18m36.814561727s) [upload=289802]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:49:51.352476979 (329h18m36.814569452s) [upload=289802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:49:51.679444680 (329h18m37.141537163s) [upload=289802]: Done uploading segments... Oct 7 06:49:51.679486539 (329h18m37.141579022s) [upload=289802]: Upload finished. err: <nil> Duration: 26.36547882s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............................................X....X.XXX.XX] duration: 1m0.003213371s FAIL (2025-10-07 06:50:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, Piece ID: Z7SESGLCN62KSTOGKVIZCS6BU7J6ZRABSFTC5NPU4FILC7C234WQ) context deadline exceeded; piecestore: (Node ID: 12mKMnBWeVnjs8QnyFeHTMX8ekvKN2UbK2W2L9KqAY4V136nKJK, Piece ID: LNPSGJ6UHLAKUNVH2YYKAWV4GBUA4M37MRVN7GISFXFRXYGYHBVQ) context deadline exceeded; piecestore: (Node ID: 1Ak8v5Vx9d87Q2CFqfJ5Nqu5Qp2rWMbgR8iqYvkYwpRVuZaAsV, Piece ID: I3US5ZBUAL5HS6MV5NTQBL4LIT7MEYTJOWA2NBAC6FFDO3MJF4UQ) context deadline exceeded; piecestore: (Node ID: 12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, Piece ID: IX6XJDLELOAYKTDS343KC3MYMW7CY367IQURZOHVSIK7ONWKKFBQ) context deadline exceeded ======LOGS====== :4186 , piece_id=L3QSARNJCXEJ3WHTQ5HN6A7FFEICZ6CSAYWN6WOK4OK2BE3LAWTA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.506532603 (329h18m12.968625086s) [noise=true , address=64.49.13.140:4505 , piece_id=S6ZAIUF6J72RJI5BIYEMHF5YF3FHOGI4C7ILKKGXN6PORSS4P2UA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.506928046 (329h18m12.969020539s) [noise=true , address=77.90.31.16:4743 , piece_id=IX6XJDLELOAYKTDS343KC3MYMW7CY367IQURZOHVSIK7ONWKKFBQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.507337095 (329h18m12.969429578s) [noise=true , address=64.49.13.154:4935 , piece_id=4PLUBGXC3DN26QLLUIRGQAUM55DJ33BRPIOEBT3JIG3H2JAGYXQQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.507345320 (329h18m12.969437803s) [noise=true , address=77.90.31.10:4539 , piece_id=Z7SESGLCN62KSTOGKVIZCS6BU7J6ZRABSFTC5NPU4FILC7C234WQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.507698523 (329h18m12.969790996s) [noise=true , address=64.49.13.137:4495 , piece_id=3HA3QKOJAXT42S53ZAIL3ITH75TUTK6GO77ZSLU5LBTHIXCDCG7A, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.508089979 (329h18m12.970182462s) [noise=true , address=45.91.116.5:4318 , piece_id=LL27R2SB2US5PBNL56PNXXGUEKH2OE7KSJOHOQVV6C5XQX6NQ2FQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.508469741 (329h18m12.970562214s) [noise=true , address=77.90.31.18:4836 , piece_id=ALV3OYY3GFBYUFN2JYPHW2TGQQQS7BDENMWHU5W4FJTLIMNQ5DQQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.508890052 (329h18m12.970982535s) [noise=true , address=77.90.31.14:4696 , piece_id=LNPSGJ6UHLAKUNVH2YYKAWV4GBUA4M37MRVN7GISFXFRXYGYHBVQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.508976396 (329h18m12.971068879s) [noise=true , address=77.90.31.11:4573 , piece_id=LDXUEYHB2FQLORISDRFFVXQDJL4AUZGCYQVLNFKN5DPUBIHI6EFA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.509368252 (329h18m12.971460735s) [noise=true , address=77.90.31.15:4714 , piece_id=RD2MFIK6IU4YV2YIVISS5ZK7MPZJFOPRP3T4BCE6WYXMECVR3KXA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.509525151 (329h18m12.971617645s) [noise=true , address=145.79.227.36:4530 , piece_id=MH7YIQMTYDWCI6YZ3YNZBVXBFRYGTQ4G4HPCJIGOA5AAPRUJKEHQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.509753105 (329h18m12.971845588s) [noise=true , address=145.79.227.33:4885 , piece_id=MX3GXJVUXEQDKC72MA23APCOZ62EQZ3HX6GC3XCNKZN3OJA7HY6A, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.510171952 (329h18m12.972264436s) [noise=true , address=77.90.31.13:4646 , piece_id=JVJ6QYZJDBRTFSCAFNWEDKGWCA7VVQCAJOMXCU6OYKA5NVOVOXMA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.510252737 (329h18m12.97234521s) [noise=true , address=77.90.31.12:4609 , piece_id=I3US5ZBUAL5HS6MV5NTQBL4LIT7MEYTJOWA2NBAC6FFDO3MJF4UQ, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:27.510556675 (329h18m12.972649158s) [noise=true , address=145.79.227.30:4770 , piece_id=CJ2IGEXVC6KO5GJVFDNTIMCTVKUR22VALAINRQVWBLFAPCF7BSAA, seg_pos={0 0}, upload=289799]: Uploading piece... Oct 7 06:49:28.947201724 (329h18m14.409294207s) [upload=289799]: Next returned nil segment Oct 7 06:49:28.947224588 (329h18m14.409317061s) [upload=289799]: Waiting for error group managing segments... Oct 7 06:49:29.322560017 (329h18m14.7846525s) [noise=true , address=64.49.13.140:4505 , piece_id=S6ZAIUF6J72RJI5BIYEMHF5YF3FHOGI4C7ILKKGXN6PORSS4P2UA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.361718040 (329h18m14.823810523s) [noise=true , address=64.49.13.154:4935 , piece_id=4PLUBGXC3DN26QLLUIRGQAUM55DJ33BRPIOEBT3JIG3H2JAGYXQQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.415500390 (329h18m14.877592863s) [noise=true , address=45.91.116.8:4058 , piece_id=RWJZRJMK7OOY4WTOCVOLNP5QZEXL466LE7O35CLOYSGSS4U64JFA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.492350216 (329h18m14.954442699s) [noise=true , address=45.91.116.13:4186 , piece_id=L3QSARNJCXEJ3WHTQ5HN6A7FFEICZ6CSAYWN6WOK4OK2BE3LAWTA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.502297657 (329h18m14.96439014s) [noise=true , address=77.90.31.13:4646 , piece_id=JVJ6QYZJDBRTFSCAFNWEDKGWCA7VVQCAJOMXCU6OYKA5NVOVOXMA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.547699380 (329h18m15.009791853s) [noise=true , address=77.90.31.15:4714 , piece_id=RD2MFIK6IU4YV2YIVISS5ZK7MPZJFOPRP3T4BCE6WYXMECVR3KXA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.549481233 (329h18m15.011573706s) [noise=true , address=45.91.116.5:4318 , piece_id=LL27R2SB2US5PBNL56PNXXGUEKH2OE7KSJOHOQVV6C5XQX6NQ2FQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.550229638 (329h18m15.012322121s) [noise=true , address=77.90.31.10:4539 , piece_id=Z7SESGLCN62KSTOGKVIZCS6BU7J6ZRABSFTC5NPU4FILC7C234WQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.583397956 (329h18m15.045490439s) [noise=true , address=77.90.31.18:4836 , piece_id=ALV3OYY3GFBYUFN2JYPHW2TGQQQS7BDENMWHU5W4FJTLIMNQ5DQQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.630661152 (329h18m15.092753635s) [noise=true , address=77.90.31.16:4743 , piece_id=IX6XJDLELOAYKTDS343KC3MYMW7CY367IQURZOHVSIK7ONWKKFBQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.871214956 (329h18m15.333307439s) [noise=true , address=77.90.31.11:4573 , piece_id=LDXUEYHB2FQLORISDRFFVXQDJL4AUZGCYQVLNFKN5DPUBIHI6EFA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.874868092 (329h18m15.336960565s) [noise=true , address=77.90.31.14:4696 , piece_id=LNPSGJ6UHLAKUNVH2YYKAWV4GBUA4M37MRVN7GISFXFRXYGYHBVQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:29.909026826 (329h18m15.371119309s) [noise=true , address=77.90.31.12:4609 , piece_id=I3US5ZBUAL5HS6MV5NTQBL4LIT7MEYTJOWA2NBAC6FFDO3MJF4UQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:49.633852117 (329h18m35.0959446s) [noise=true , address=64.49.13.137:4495 , piece_id=3HA3QKOJAXT42S53ZAIL3ITH75TUTK6GO77ZSLU5LBTHIXCDCG7A, seg_pos={0 0}, upload=289799]: Done uploading piece. err: <nil> Oct 7 06:49:49.633880160 (329h18m35.095972633s) [seg_pos={0 0}, upload=289799]: Segment reached optimal threshold of 14 pieces. Oct 7 06:49:49.634195380 (329h18m35.096287863s) [noise=true , address=145.79.227.30:4770 , piece_id=CJ2IGEXVC6KO5GJVFDNTIMCTVKUR22VALAINRQVWBLFAPCF7BSAA, seg_pos={0 0}, upload=289799]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Vpr5UhYBKjdwam6GJFcMqrwjdkaCstSTrxgX8AzX5VjhWotK5): protocol: read tcp 188.166.225.170:40606->145.79.227.30:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:49.634206021 (329h18m35.096298504s) [noise=true , address=145.79.227.30:4770 , piece_id=CJ2IGEXVC6KO5GJVFDNTIMCTVKUR22VALAINRQVWBLFAPCF7BSAA, seg_pos={0 0}, upload=289799]: Long tail cancellation due to optimal threshold Oct 7 06:49:49.634212303 (329h18m35.096304796s) [noise=true , address=145.79.227.33:4885 , piece_id=MX3GXJVUXEQDKC72MA23APCOZ62EQZ3HX6GC3XCNKZN3OJA7HY6A, seg_pos={0 0}, upload=289799]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: rpc: dial tcp 145.79.227.33:4885: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:49.634226219 (329h18m35.096318702s) [noise=true , address=145.79.227.33:4885 , piece_id=MX3GXJVUXEQDKC72MA23APCOZ62EQZ3HX6GC3XCNKZN3OJA7HY6A, seg_pos={0 0}, upload=289799]: Long tail cancellation due to optimal threshold Oct 7 06:49:49.634301583 (329h18m35.096394056s) [noise=true , address=145.79.227.37:4568 , piece_id=IRMG732T746T7R4CLIGSUUVMNXIG77SI2PG63KHLCLK2ZJ6SL6JQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zJigHghNntzHNWim5RnWcMPfeUAbqsrvrfWretYcyEkXbSYzJ): protocol: read tcp 188.166.225.170:54414->145.79.227.37:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:49.634308065 (329h18m35.096400549s) [noise=true , address=145.79.227.37:4568 , piece_id=IRMG732T746T7R4CLIGSUUVMNXIG77SI2PG63KHLCLK2ZJ6SL6JQ, seg_pos={0 0}, upload=289799]: Long tail cancellation due to optimal threshold Oct 7 06:49:49.634422303 (329h18m35.096514786s) [noise=true , address=145.79.227.36:4530 , piece_id=MH7YIQMTYDWCI6YZ3YNZBVXBFRYGTQ4G4HPCJIGOA5AAPRUJKEHQ, seg_pos={0 0}, upload=289799]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WVxMFx9jm6CT7GcXg2qgq3Be9UPFf1KZKssqufSk2ebdBBmxR): protocol: context canceled; context canceled Oct 7 06:49:49.634431520 (329h18m35.096524003s) [noise=true , address=145.79.227.36:4530 , piece_id=MH7YIQMTYDWCI6YZ3YNZBVXBFRYGTQ4G4HPCJIGOA5AAPRUJKEHQ, seg_pos={0 0}, upload=289799]: Long tail cancellation due to optimal threshold Oct 7 06:49:49.634475224 (329h18m35.096567707s) [seg_pos={0 0}, upload=289799]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:49:49.634588459 (329h18m35.096680942s) [upload=289799]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:49:49.634595633 (329h18m35.096688116s) [upload=289799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:49:49.962962507 (329h18m35.42505498s) [upload=289799]: Done uploading segments... Oct 7 06:49:49.962980761 (329h18m35.425073244s) [upload=289799]: Upload finished. err: <nil> Duration: 1m0.003231037s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................................X.....XXXXXXXX.XXX] duration: 30.002467701s FAIL (2025-10-07 06:50:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, Piece ID: B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A) context deadline exceeded ======LOGS====== load=289802]: Starting upload Oct 7 06:49:43.013416725 (329h18m28.475509208s) [upload=289802]: Done starting upload Oct 7 06:49:43.013521785 (329h18m28.475614258s) [upload=289802]: Uploading segments... Oct 7 06:49:43.013671580 (329h18m28.475764063s) [upload=289802]: Got next segment. Inline: false Oct 7 06:49:43.013748838 (329h18m28.475841311s) [upload=289802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:49:43.013756813 (329h18m28.475849316s) [upload=289802]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:49:43.301776509 (329h18m28.763868992s) [seg_pos={0 0}, upload=289802]: Begin upload segment... Oct 7 06:49:43.301892891 (329h18m28.763985374s) [seg_pos={0 0}, upload=289802]: Done begin upload segment. Oct 7 06:49:43.301914141 (329h18m28.764006614s) [noise=true , address=45.91.116.9:4262 , piece_id=B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.301925783 (329h18m28.764018256s) [noise=true , address=77.90.31.10:4517 , piece_id=I3IXWV4LB57CPF7DWZR5WHDE4WCLTEWPMLVLENYK7PJCKOPUNL4A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302003843 (329h18m28.764096326s) [noise=true , address=77.90.31.18:4805 , piece_id=7GXSRX6YF3NXI7KKDZNEXW55O564OJPCMMLZS5NKV5FPZJYOGNWA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302532549 (329h18m28.764625032s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.302987566 (329h18m28.765080049s) [noise=true , address=45.91.116.13:4155 , piece_id=B3TIGHP4NK4OI442VHX4HKJN3ZALOGMAVPHS5MB44TXTULMWUW6A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.303281294 (329h18m28.765373777s) [noise=true , address=45.91.116.5:4298 , piece_id=RMKPPLJGNJ3H5QBOXWVBMRJ6ECFEZCAT77XGWEQF7ZCNYMK7RNQQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.303691746 (329h18m28.765784229s) [noise=true , address=77.90.31.11:4555 , piece_id=4TNNCKUFE7OJIFXDTQUXZB4U6AKL2TWWS32YTJVINFFO7AOBTE2Q, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.305942041 (329h18m28.768034535s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.306481458 (329h18m28.768573941s) [noise=true , address=45.91.116.4:4937 , piece_id=M3NYCLHYZH6MNIEH3UWWAQWX7XWAWQX6CIXBFOSXS46V6MIHHCMA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.306995507 (329h18m28.76908799s) [noise=true , address=45.91.116.11:4117 , piece_id=JCZ23FW2MQKQ6G6UPLTLMDSATKPNXPNTFPXNXAHZC4RWDHCIZ6PA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.308596987 (329h18m28.77068947s) [noise=true , address=45.91.116.6:4975 , piece_id=XFBC2BPMDTSUWBXIDUBYSTCEQ4D4DXZKXNMF5BQ446FPAIYMHLSA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.310241648 (329h18m28.772334121s) [noise=true , address=77.90.31.10:4518 , piece_id=AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.311645512 (329h18m28.773737995s) [noise=true , address=77.90.31.14:4661 , piece_id=GCFDREABYIPQMFULQLL25B2UEANXVGXVWUJAB3S6M4OCRNGCR27A, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.313248414 (329h18m28.775340897s) [noise=true , address=45.91.116.4:4940 , piece_id=5RYKMI6YCCYT3KLZFONVARPDRRIG6DBSJA6UQUVSVK4EBQJ6FHNA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.314782415 (329h18m28.776874888s) [noise=true , address=45.91.116.13:4154 , piece_id=4JKN275HNOUS67UHI6DFS7RYHDEKG5FQD4NJ4CY4XHCDENP65HYA, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:43.314970062 (329h18m28.777062545s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Uploading piece... Oct 7 06:49:45.075789771 (329h18m30.537882284s) [upload=289802]: Next returned nil segment Oct 7 06:49:45.075816292 (329h18m30.537908795s) [upload=289802]: Waiting for error group managing segments... Oct 7 06:49:45.376070605 (329h18m30.838163088s) [noise=true , address=77.90.31.11:4555 , piece_id=4TNNCKUFE7OJIFXDTQUXZB4U6AKL2TWWS32YTJVINFFO7AOBTE2Q, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.416112341 (329h18m30.878204814s) [noise=true , address=45.91.116.5:4298 , piece_id=RMKPPLJGNJ3H5QBOXWVBMRJ6ECFEZCAT77XGWEQF7ZCNYMK7RNQQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.427448547 (329h18m30.88954102s) [noise=true , address=45.91.116.4:4937 , piece_id=M3NYCLHYZH6MNIEH3UWWAQWX7XWAWQX6CIXBFOSXS46V6MIHHCMA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.448475256 (329h18m30.910567729s) [noise=true , address=45.91.116.13:4154 , piece_id=4JKN275HNOUS67UHI6DFS7RYHDEKG5FQD4NJ4CY4XHCDENP65HYA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.482005883 (329h18m30.944098357s) [noise=true , address=45.91.116.6:4975 , piece_id=XFBC2BPMDTSUWBXIDUBYSTCEQ4D4DXZKXNMF5BQ446FPAIYMHLSA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.482247453 (329h18m30.944339936s) [noise=true , address=77.90.31.14:4661 , piece_id=GCFDREABYIPQMFULQLL25B2UEANXVGXVWUJAB3S6M4OCRNGCR27A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.490005565 (329h18m30.952098048s) [noise=true , address=77.90.31.10:4518 , piece_id=AIHMINRIYSB3M3TK6CO7UNDGS75VDUB3PPXCLOF3Q6JKHE3TBQ2A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.496317425 (329h18m30.958409908s) [noise=true , address=45.91.116.4:4940 , piece_id=5RYKMI6YCCYT3KLZFONVARPDRRIG6DBSJA6UQUVSVK4EBQJ6FHNA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.500978250 (329h18m30.963070733s) [noise=true , address=77.90.31.18:4805 , piece_id=7GXSRX6YF3NXI7KKDZNEXW55O564OJPCMMLZS5NKV5FPZJYOGNWA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.515497326 (329h18m30.977589799s) [noise=true , address=45.91.116.13:4155 , piece_id=B3TIGHP4NK4OI442VHX4HKJN3ZALOGMAVPHS5MB44TXTULMWUW6A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:45.849733706 (329h18m31.311826179s) [noise=true , address=45.91.116.9:4262 , piece_id=B4EVXY5MYVAFJAUZ3IX327WG4XHQEIXFNDOVEHCA7C2572R6KQBA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:46.032133195 (329h18m31.494225669s) [noise=true , address=77.90.31.10:4517 , piece_id=I3IXWV4LB57CPF7DWZR5WHDE4WCLTEWPMLVLENYK7PJCKOPUNL4A, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:51.351895502 (329h18m36.813987975s) [noise=true , address=45.91.116.11:4117 , piece_id=JCZ23FW2MQKQ6G6UPLTLMDSATKPNXPNTFPXNXAHZC4RWDHCIZ6PA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: <nil> Oct 7 06:49:51.351933755 (329h18m36.814026238s) [seg_pos={0 0}, upload=289802]: Segment reached optimal threshold of 13 pieces. Oct 7 06:49:51.352216383 (329h18m36.814308856s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 145.79.227.14:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352226461 (329h18m36.814318935s) [noise=true , address=145.79.227.14:4840 , piece_id=MNR27HTUWBXJXAV5BJUDUYLXXYQFU3YYUVA5AXRDFDARP47XTLUQ, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352248384 (329h18m36.814340867s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: read tcp 188.166.225.170:56122->145.79.227.5:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352257280 (329h18m36.814349763s) [noise=true , address=145.79.227.5:4222 , piece_id=56VPY7YLCQE2BLTJH36LVZHNUXM2YE3SRXJ2VW5CF7ZLPVLRDVBA, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352361849 (329h18m36.814454353s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: read tcp 188.166.225.170:57450->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:51.352371328 (329h18m36.814463821s) [noise=true , address=145.79.227.13:4803 , piece_id=TWEDZV3DMOQGVMSGNOCDPBKII67WRWUQLSITULPZUL3VTWIMSQKQ, seg_pos={0 0}, upload=289802]: Long tail cancellation due to optimal threshold Oct 7 06:49:51.352391606 (329h18m36.814484079s) [seg_pos={0 0}, upload=289802]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:49:51.352469234 (329h18m36.814561727s) [upload=289802]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:49:51.352476979 (329h18m36.814569452s) [upload=289802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:49:51.679444680 (329h18m37.141537163s) [upload=289802]: Done uploading segments... Oct 7 06:49:51.679486539 (329h18m37.141579022s) [upload=289802]: Upload finished. err: <nil> Duration: 30.002486848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................X.....XXXXXXXX.XX] duration: 30.002931309s FAIL (2025-10-07 06:49:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=289795]: Starting upload Oct 7 06:49:12.783921973 (329h17m58.246014446s) [upload=289795]: Done starting upload Oct 7 06:49:12.784067660 (329h17m58.246160143s) [upload=289795]: Uploading segments... Oct 7 06:49:12.784571840 (329h17m58.246664323s) [upload=289795]: Got next segment. Inline: false Oct 7 06:49:12.784632326 (329h17m58.246724809s) [upload=289795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:49:12.784830633 (329h17m58.246923116s) [upload=289795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:49:13.102321047 (329h17m58.56441354s) [seg_pos={0 0}, upload=289795]: Begin upload segment... Oct 7 06:49:13.102439001 (329h17m58.564531494s) [seg_pos={0 0}, upload=289795]: Done begin upload segment. Oct 7 06:49:13.102477114 (329h17m58.564569588s) [noise=true , address=77.90.31.16:4734 , piece_id=A2O2YZKGFZA3WVXR45IBJFIP4Z3CKKNGRZMJDLSHB43UPFTS6EQQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.102536327 (329h17m58.56462881s) [noise=true , address=77.90.31.11:4555 , piece_id=M74HSDZZRSZ2SGYWO2XPIYXVQWAMYIQN4DUQQOCJIZHVWP4S4AOQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.103463633 (329h17m58.565556146s) [noise=true , address=45.91.116.4:4938 , piece_id=NJOUZPVFWWOHWRZNV5X66B63J6PY44QPTJCPOXPEDJ4N5K44UJOA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.104136383 (329h17m58.566228866s) [noise=true , address=45.91.116.6:4974 , piece_id=3RLEDEO5BQ7MFRGDIVWNQES4PFLG7TCY4RQXFRP5PHBFG3U3KTKA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.104149319 (329h17m58.566241802s) [noise=true , address=45.91.116.11:4119 , piece_id=DQSX5OMF24IP7HT2JTTNQHLRBYCIBF4L2JPOM5W7LS7HA7MGF5PA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.104571272 (329h17m58.566663755s) [noise=true , address=145.79.227.17:4971 , piece_id=H5PIBVBR2AEWONBUQVA642VBAPSWJULMHQAXVFLAQMHT3RMBRGQQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.104591139 (329h17m58.566683622s) [noise=true , address=45.91.116.5:4299 , piece_id=TQGXJ72C77M7WKOOL6G6IF2GGJPXJLFJ6ORYGQEYNKIDFA4N34UA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.105098586 (329h17m58.567191069s) [noise=true , address=45.91.116.7:4011 , piece_id=2OV2OZYZZ47ZJHQNKETRVLJDGBXZWKCQXN5GONZLMPOZ4X7BYIFQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.105112262 (329h17m58.567204745s) [noise=true , address=45.91.116.14:4189 , piece_id=77QYG6YUF6OCLBAU4YWGJZIAKHUAKBFFCB56LVC43DCO4YBO6I4A, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.105575715 (329h17m58.567668198s) [noise=true , address=145.79.227.16:4911 , piece_id=FJQ64NQK4N7UKFL7BCIDPQYG7ZGEDGQNE36I37YY7K3B2NSK2IBQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.105858683 (329h17m58.567951197s) [noise=true , address=77.90.31.21:4879 , piece_id=VOEQRTAWPPHEPU7SEB43JJAPICH26JQIIKMO6NW5EJXQUXOB2PCA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.105993771 (329h17m58.568086254s) [noise=true , address=45.91.116.11:4118 , piece_id=34BCZGJJ7KORQGHMZJ5INSVEHDD6S2ELXITTGPCCJPDVOQGCD7LA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.106358705 (329h17m58.568451188s) [noise=true , address=145.79.227.18:4007 , piece_id=6NJJN2PWWIVZQM6E4VVUCQBFIFLHG5YXZIPOPZ4CWDGA3QWUGSVQ, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.106449889 (329h17m58.568542372s) [noise=true , address=45.91.116.9:4263 , piece_id=YR6HGLBMHAHK3W2ZUMS5FOL6EIOYQ7QWLISBLQVGY3D4KBB2ATQA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.106868396 (329h17m58.568960879s) [noise=true , address=77.90.31.17:4771 , piece_id=4HOS6TSEIAONBKSK2C7OWA2SYGJU4DQYSYYTLPZMTEWKQVABJD7Q, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:13.106889235 (329h17m58.568981728s) [noise=true , address=77.90.31.21:4878 , piece_id=KLF6NUPUYARPRNTIQQ5DLHJXV76KZDMXFYHDE7PX76UUQOONNNUA, seg_pos={0 0}, upload=289795]: Uploading piece... Oct 7 06:49:14.767438292 (329h18m0.229530765s) [upload=289795]: Next returned nil segment Oct 7 06:49:14.767461577 (329h18m0.22955406s) [upload=289795]: Waiting for error group managing segments... Oct 7 06:49:15.161656187 (329h18m0.62374867s) [noise=true , address=77.90.31.21:4878 , piece_id=KLF6NUPUYARPRNTIQQ5DLHJXV76KZDMXFYHDE7PX76UUQOONNNUA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.169518909 (329h18m0.631611393s) [noise=true , address=45.91.116.4:4938 , piece_id=NJOUZPVFWWOHWRZNV5X66B63J6PY44QPTJCPOXPEDJ4N5K44UJOA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.184826819 (329h18m0.646919292s) [noise=true , address=45.91.116.9:4263 , piece_id=YR6HGLBMHAHK3W2ZUMS5FOL6EIOYQ7QWLISBLQVGY3D4KBB2ATQA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.214029364 (329h18m0.676121848s) [noise=true , address=45.91.116.14:4189 , piece_id=77QYG6YUF6OCLBAU4YWGJZIAKHUAKBFFCB56LVC43DCO4YBO6I4A, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.217835392 (329h18m0.679927865s) [noise=true , address=45.91.116.11:4118 , piece_id=34BCZGJJ7KORQGHMZJ5INSVEHDD6S2ELXITTGPCCJPDVOQGCD7LA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.266545875 (329h18m0.728638348s) [noise=true , address=77.90.31.17:4771 , piece_id=4HOS6TSEIAONBKSK2C7OWA2SYGJU4DQYSYYTLPZMTEWKQVABJD7Q, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.300992968 (329h18m0.763085451s) [noise=true , address=45.91.116.6:4974 , piece_id=3RLEDEO5BQ7MFRGDIVWNQES4PFLG7TCY4RQXFRP5PHBFG3U3KTKA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.319458810 (329h18m0.781551283s) [noise=true , address=45.91.116.7:4011 , piece_id=2OV2OZYZZ47ZJHQNKETRVLJDGBXZWKCQXN5GONZLMPOZ4X7BYIFQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.380145257 (329h18m0.84223773s) [noise=true , address=77.90.31.21:4879 , piece_id=VOEQRTAWPPHEPU7SEB43JJAPICH26JQIIKMO6NW5EJXQUXOB2PCA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:15.688597701 (329h18m1.150690184s) [noise=true , address=77.90.31.16:4734 , piece_id=A2O2YZKGFZA3WVXR45IBJFIP4Z3CKKNGRZMJDLSHB43UPFTS6EQQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:16.047692936 (329h18m1.509785419s) [noise=true , address=45.91.116.5:4299 , piece_id=TQGXJ72C77M7WKOOL6G6IF2GGJPXJLFJ6ORYGQEYNKIDFA4N34UA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:16.849618634 (329h18m2.311711107s) [noise=true , address=77.90.31.11:4555 , piece_id=M74HSDZZRSZ2SGYWO2XPIYXVQWAMYIQN4DUQQOCJIZHVWP4S4AOQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:21.596764507 (329h18m7.05885699s) [noise=true , address=45.91.116.11:4119 , piece_id=DQSX5OMF24IP7HT2JTTNQHLRBYCIBF4L2JPOM5W7LS7HA7MGF5PA, seg_pos={0 0}, upload=289795]: Done uploading piece. err: <nil> Oct 7 06:49:21.596887982 (329h18m7.058980465s) [seg_pos={0 0}, upload=289795]: Segment reached optimal threshold of 13 pieces. Oct 7 06:49:21.597181531 (329h18m7.059274014s) [noise=true , address=145.79.227.18:4007 , piece_id=6NJJN2PWWIVZQM6E4VVUCQBFIFLHG5YXZIPOPZ4CWDGA3QWUGSVQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: read tcp 188.166.225.170:36722->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:21.597191751 (329h18m7.059284234s) [noise=true , address=145.79.227.18:4007 , piece_id=6NJJN2PWWIVZQM6E4VVUCQBFIFLHG5YXZIPOPZ4CWDGA3QWUGSVQ, seg_pos={0 0}, upload=289795]: Long tail cancellation due to optimal threshold Oct 7 06:49:21.597252657 (329h18m7.05934514s) [noise=true , address=145.79.227.17:4971 , piece_id=H5PIBVBR2AEWONBUQVA642VBAPSWJULMHQAXVFLAQMHT3RMBRGQQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: read tcp 188.166.225.170:55532->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:21.597265131 (329h18m7.059357614s) [noise=true , address=145.79.227.17:4971 , piece_id=H5PIBVBR2AEWONBUQVA642VBAPSWJULMHQAXVFLAQMHT3RMBRGQQ, seg_pos={0 0}, upload=289795]: Long tail cancellation due to optimal threshold Oct 7 06:49:21.597315296 (329h18m7.059407779s) [noise=true , address=145.79.227.16:4911 , piece_id=FJQ64NQK4N7UKFL7BCIDPQYG7ZGEDGQNE36I37YY7K3B2NSK2IBQ, seg_pos={0 0}, upload=289795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: read tcp 188.166.225.170:42302->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:21.597324304 (329h18m7.059416777s) [noise=true , address=145.79.227.16:4911 , piece_id=FJQ64NQK4N7UKFL7BCIDPQYG7ZGEDGQNE36I37YY7K3B2NSK2IBQ, seg_pos={0 0}, upload=289795]: Long tail cancellation due to optimal threshold Oct 7 06:49:21.597342608 (329h18m7.059435091s) [seg_pos={0 0}, upload=289795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:49:21.597424443 (329h18m7.059516927s) [upload=289795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:49:21.597431408 (329h18m7.059523881s) [upload=289795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:49:21.906404453 (329h18m7.368496936s) [upload=289795]: Done uploading segments... Oct 7 06:49:21.906427696 (329h18m7.368520179s) [upload=289795]: Upload finished. err: <nil> Duration: 30.002944254s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................X....X.XXX.X] duration: 1m0.247380536s FAIL (2025-10-07 06:49:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== (329h17m12.199945541s) [upload=289784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:48:26.737859410 (329h17m12.199951893s) [upload=289784]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:48:27.017460947 (329h17m12.47955343s) [seg_pos={0 0}, upload=289784]: Begin upload segment... Oct 7 06:48:27.017524909 (329h17m12.479617382s) [seg_pos={0 0}, upload=289784]: Done begin upload segment. Oct 7 06:48:27.017540648 (329h17m12.479633131s) [noise=true , address=145.79.227.23:4567 , piece_id=PMXY6QCL2ZZYRT7KYQB5AWDLUJD7SUYDQHL3OCFI52YIND7VK44Q, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.017569914 (329h17m12.479662397s) [noise=true , address=45.91.116.14:4210 , piece_id=T3IE6KPQQJ3UXPEKBC4CVDSXRS4HWYZRH3BSJEVGSGKHDYML2DHQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.017619068 (329h17m12.479711551s) [noise=true , address=145.79.227.24:4581 , piece_id=M23PAO2E6PNKZDOF7SDIFGE6SLXT4P2FWGUDAE7S6SW4R63TG5PQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.018252023 (329h17m12.480344516s) [noise=true , address=145.79.227.28:4732 , piece_id=RPB3ZO72V74CURCP3X2ZAY77EPQS32CW4T7O3P3VPPJ3GRBG6SAQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.018703142 (329h17m12.480795625s) [noise=true , address=77.90.31.18:4824 , piece_id=DC7I25ZPOTRLXM75E3Q7Q565UG5U3B5566LYA7RUEWGUGCSE4QYA, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.019221980 (329h17m12.481314463s) [noise=true , address=45.91.116.5:4302 , piece_id=W7XJJDZH65CO42WP6KBJTXFWSMYYWT3K7M3M244ME2QXZFFRWRWA, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.019681976 (329h17m12.481774459s) [noise=true , address=77.90.31.15:4724 , piece_id=HAJMJYLYTTJC5OKWLSS62LIACG62YVVTCXT2FRBBXFZC5U72PO2A, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.020181707 (329h17m12.48227419s) [noise=true , address=77.90.31.19:4850 , piece_id=D4RJESYXMQVNH2FX3UMYLKQOAIJ7FVDYR2GSHAXQ6FDNDR3EPYEQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.020774897 (329h17m12.48286738s) [noise=true , address=145.79.227.33:4870 , piece_id=LJDASID6JZOU7GAGFRRCDXU4CUWJ24V434RKEQEXI2LJUNNPOBEQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.021267304 (329h17m12.483359777s) [noise=true , address=77.90.31.13:4637 , piece_id=ZOY4RLKKRP7GA7WG3SGHOXJE4OWERH33TNXFONHOKPCDVUDS26NA, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.021882937 (329h17m12.48397542s) [noise=true , address=77.90.31.16:4739 , piece_id=EFFABAGLH7XON7JOFVD6ZTDCVNK6DFT7QGQVHE5VGYBHUJIYYE7Q, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.022086034 (329h17m12.484178517s) [noise=true , address=45.91.116.9:4279 , piece_id=IWVQ47JZZPLR6UCQLHPRZUAWVFOUVRXWR6B5NEMZMP5SAZU5WZSQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.022613889 (329h17m12.484706362s) [noise=true , address=45.91.116.11:4128 , piece_id=VKKQRKFJHPB5WQELFZYAQDKH7YP26OS7HUIU7O2WYYJWGRTXB52Q, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.024128203 (329h17m12.486220686s) [noise=true , address=77.90.31.10:4543 , piece_id=47SCIGDSLELWW6B6X4WJYML4MVM7XRINYXXDJPCF5QJPWNL6F76A, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.024770104 (329h17m12.486862588s) [noise=true , address=77.90.31.14:4667 , piece_id=CCONERHE4QJNBEYN2CFJELL7XPFABXHFA5NZS3ZDYJNFBRVE7AQA, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.024994102 (329h17m12.487086585s) [noise=true , address=45.91.116.7:4013 , piece_id=KY23OFTBZKMC3OIZCHAHUE3GTNEWLXT2RQOW3WKAIL55END4JPYA, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.025645812 (329h17m12.487738295s) [noise=true , address=145.79.227.44:4197 , piece_id=IA4KK3QT26TTKTWBIVDPGDQA2WX76JPVFB6IHYBS7FWHMZSHUKYQ, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:27.026353700 (329h17m12.488446194s) [noise=true , address=45.91.116.8:4054 , piece_id=AMBV7ZMOVL5DEQSVXMZV5JJKQJJTEVXG3IPVW77FLOVAD7WUHG4A, seg_pos={0 0}, upload=289784]: Uploading piece... Oct 7 06:48:28.542053822 (329h17m14.004146305s) [upload=289784]: Next returned nil segment Oct 7 06:48:28.542073729 (329h17m14.004166212s) [upload=289784]: Waiting for error group managing segments... Oct 7 06:48:28.928307332 (329h17m14.390399815s) [noise=true , address=77.90.31.19:4850 , piece_id=D4RJESYXMQVNH2FX3UMYLKQOAIJ7FVDYR2GSHAXQ6FDNDR3EPYEQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:28.944939081 (329h17m14.407031565s) [noise=true , address=45.91.116.9:4279 , piece_id=IWVQ47JZZPLR6UCQLHPRZUAWVFOUVRXWR6B5NEMZMP5SAZU5WZSQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:28.973893585 (329h17m14.435986068s) [noise=true , address=45.91.116.7:4013 , piece_id=KY23OFTBZKMC3OIZCHAHUE3GTNEWLXT2RQOW3WKAIL55END4JPYA, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:28.999868226 (329h17m14.461960709s) [noise=true , address=45.91.116.5:4302 , piece_id=W7XJJDZH65CO42WP6KBJTXFWSMYYWT3K7M3M244ME2QXZFFRWRWA, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.016140651 (329h17m14.478233134s) [noise=true , address=77.90.31.16:4739 , piece_id=EFFABAGLH7XON7JOFVD6ZTDCVNK6DFT7QGQVHE5VGYBHUJIYYE7Q, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.024344462 (329h17m14.486436955s) [noise=true , address=45.91.116.8:4054 , piece_id=AMBV7ZMOVL5DEQSVXMZV5JJKQJJTEVXG3IPVW77FLOVAD7WUHG4A, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.032134516 (329h17m14.494226999s) [noise=true , address=77.90.31.10:4543 , piece_id=47SCIGDSLELWW6B6X4WJYML4MVM7XRINYXXDJPCF5QJPWNL6F76A, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.033422719 (329h17m14.495515202s) [noise=true , address=77.90.31.18:4824 , piece_id=DC7I25ZPOTRLXM75E3Q7Q565UG5U3B5566LYA7RUEWGUGCSE4QYA, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.057097830 (329h17m14.519190313s) [noise=true , address=77.90.31.13:4637 , piece_id=ZOY4RLKKRP7GA7WG3SGHOXJE4OWERH33TNXFONHOKPCDVUDS26NA, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.065762780 (329h17m14.527855263s) [noise=true , address=45.91.116.14:4210 , piece_id=T3IE6KPQQJ3UXPEKBC4CVDSXRS4HWYZRH3BSJEVGSGKHDYML2DHQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.077914268 (329h17m14.540006751s) [noise=true , address=45.91.116.11:4128 , piece_id=VKKQRKFJHPB5WQELFZYAQDKH7YP26OS7HUIU7O2WYYJWGRTXB52Q, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.601175363 (329h17m15.063267846s) [noise=true , address=77.90.31.14:4667 , piece_id=CCONERHE4QJNBEYN2CFJELL7XPFABXHFA5NZS3ZDYJNFBRVE7AQA, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:48:29.811249658 (329h17m15.273342141s) [noise=true , address=77.90.31.15:4724 , piece_id=HAJMJYLYTTJC5OKWLSS62LIACG62YVVTCXT2FRBBXFZC5U72PO2A, seg_pos={0 0}, upload=289784]: Done uploading piece. err: <nil> Oct 7 06:49:26.031189208 (329h18m11.493281681s) [noise=true , address=145.79.227.28:4732 , piece_id=RPB3ZO72V74CURCP3X2ZAY77EPQS32CW4T7O3P3VPPJ3GRBG6SAQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: ecclient: upload failed (node:171oGgyDi35YSen7MKLmHUeQjZKwNAoYt5AX4D34YrHaBEoiLH, address:145.79.227.28:4732): protocol: rpc: dial tcp 145.79.227.28:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:26.031324366 (329h18m11.493416839s) [noise=true , address=145.79.227.24:4581 , piece_id=M23PAO2E6PNKZDOF7SDIFGE6SLXT4P2FWGUDAE7S6SW4R63TG5PQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: read tcp 188.166.225.170:51920->145.79.227.24:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:26.031337901 (329h18m11.493430374s) [noise=true , address=145.79.227.44:4197 , piece_id=IA4KK3QT26TTKTWBIVDPGDQA2WX76JPVFB6IHYBS7FWHMZSHUKYQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: ecclient: upload failed (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi, address:145.79.227.44:4197): protocol: read tcp 188.166.225.170:39924->145.79.227.44:4197: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:26.031398537 (329h18m11.49349101s) [noise=true , address=145.79.227.23:4567 , piece_id=PMXY6QCL2ZZYRT7KYQB5AWDLUJD7SUYDQHL3OCFI52YIND7VK44Q, seg_pos={0 0}, upload=289784]: Done uploading piece. err: ecclient: upload failed (node:12CpXY398qxh5vfhGabZ5WSs7xgRG8R27oDkKZfqBzzwhf4NxtY, address:145.79.227.23:4567): protocol: read tcp 188.166.225.170:54456->145.79.227.23:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:26.031451929 (329h18m11.493544412s) [noise=true , address=145.79.227.33:4870 , piece_id=LJDASID6JZOU7GAGFRRCDXU4CUWJ24V434RKEQEXI2LJUNNPOBEQ, seg_pos={0 0}, upload=289784]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: read tcp 188.166.225.170:52236->145.79.227.33:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:26.031481174 (329h18m11.493573647s) [seg_pos={0 0}, upload=289784]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:49:26.276430348 (329h18m11.738522831s) [upload=289784]: Done uploading segments... Oct 7 06:49:26.276461868 (329h18m11.738554351s) [upload=289784]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.247416074s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............................................X.....XXXXXXXX.X] duration: 30.249419999s FAIL (2025-10-07 06:49:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 89786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:48:42.606613598 (329h17m28.068706071s) [seg_pos={0 0}, upload=289786]: Begin upload segment... Oct 7 06:48:42.606700092 (329h17m28.068792575s) [seg_pos={0 0}, upload=289786]: Done begin upload segment. Oct 7 06:48:42.606740098 (329h17m28.068832581s) [noise=true , address=145.79.227.17:4971 , piece_id=52Z3SIGTDQBFFJRS2VB6M7Z3PHHUJH54UOZCP6BHC6UMXXBEUK4Q, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.609949610 (329h17m28.072042093s) [noise=true , address=45.91.116.10:4082 , piece_id=H2IXIICLC5YV47PUVUB7GUEISTJCIVGMQMS62U4XIKK4RC6R3H3A, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.610520357 (329h17m28.07261284s) [noise=true , address=145.79.227.7:4295 , piece_id=LS57LC7KZKRSUNUYTROTOY6TNTMZBRR4772YPEBAJH4AYHSL5LAQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.610992186 (329h17m28.073084689s) [noise=true , address=145.79.227.15:4873 , piece_id=FOI5IVBTLCOKQFZVPDIQQML42TD27DK23AWBBIHJPET7VPHMQ4YQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.611424398 (329h17m28.073516881s) [noise=true , address=145.79.227.14:4840 , piece_id=F72327RHJVZPDANSVTTUGMPPGI3QHWXKMGH6GYJBFHZFW5LC7YRQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.612056602 (329h17m28.074149085s) [noise=true , address=77.90.31.14:4661 , piece_id=3VGGQRAXFR7WWXUPXTCON5Q7COQ5P3RIMXX4W6RM5AENTTJ4URIQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.612428310 (329h17m28.074520793s) [noise=true , address=145.79.227.4:4186 , piece_id=JPND4ZFPQJJ53RDLJUAXKJKOBG32RQQHZT7V3IRDFCKE3AFSWGUA, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.612791531 (329h17m28.074884014s) [noise=true , address=45.91.116.12:4226 , piece_id=72SBOB5LEZONGYXE72IWV5JINZILPJHVZFR3KIEMFATQ4PDHEI7A, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.613199719 (329h17m28.075292202s) [noise=true , address=45.91.116.8:4048 , piece_id=RKO7Q4NHBBFUM2JGH62ABYMNJKIIFZ7GN2GPWFV6363ASRJEKQ5A, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.613689040 (329h17m28.075781513s) [noise=true , address=145.79.227.14:4838 , piece_id=3R2NSXPFQW4SXVX4H6LLETXOKKIYUBQWFBLEONHGXUCK2SKA673A, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.614247624 (329h17m28.076340107s) [noise=true , address=77.90.31.16:4733 , piece_id=4DTJ45M3LSMTLKL7WPEDPBJZQQLTZNQJH5VLWMCGZHXFAULZ4ZLA, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.617069848 (329h17m28.079162331s) [noise=true , address=77.90.31.13:4627 , piece_id=D2Z5T6MLPZ2ENZJI5EOHAMUUBQSDT3V7E5YKZWXALGTQNTJLHK3Q, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.617714746 (329h17m28.079807219s) [noise=true , address=145.79.227.2:4113 , piece_id=CDZE7JG62NREH2PATTQSLBEBCPFKUDG2TKEL2RCMATSNRJOBUWTQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.619368665 (329h17m28.081461148s) [noise=true , address=145.79.227.11:4730 , piece_id=CWXE2NNRGALUVRQGYVEXCV3R5FXGCJ7QYAGEYRJPBJ6V6G7BHZYQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.621064955 (329h17m28.083157439s) [noise=true , address=145.79.227.3:4151 , piece_id=IN4WIZE6I4TKHQB2UJYHP52T7JPG2ADU2QQ6WGDFFKBRRUYHNRKQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:42.622723284 (329h17m28.084815797s) [noise=true , address=45.91.116.4:4937 , piece_id=T3U7JDJZ7NONMSTGPNQ4LAZHDRD3AJTPEVDX5KONSESNFFBAGMKQ, seg_pos={0 0}, upload=289786]: Uploading piece... Oct 7 06:48:44.405986660 (329h17m29.868079143s) [upload=289786]: Next returned nil segment Oct 7 06:48:44.406011787 (329h17m29.8681043s) [upload=289786]: Waiting for error group managing segments... Oct 7 06:48:44.689636705 (329h17m30.151729188s) [noise=true , address=77.90.31.13:4627 , piece_id=D2Z5T6MLPZ2ENZJI5EOHAMUUBQSDT3V7E5YKZWXALGTQNTJLHK3Q, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:44.720511676 (329h17m30.182604159s) [noise=true , address=45.91.116.10:4082 , piece_id=H2IXIICLC5YV47PUVUB7GUEISTJCIVGMQMS62U4XIKK4RC6R3H3A, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:44.771903465 (329h17m30.233995948s) [noise=true , address=45.91.116.12:4226 , piece_id=72SBOB5LEZONGYXE72IWV5JINZILPJHVZFR3KIEMFATQ4PDHEI7A, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:44.829241034 (329h17m30.291333517s) [noise=true , address=45.91.116.8:4048 , piece_id=RKO7Q4NHBBFUM2JGH62ABYMNJKIIFZ7GN2GPWFV6363ASRJEKQ5A, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:44.833831085 (329h17m30.295923558s) [noise=true , address=77.90.31.16:4733 , piece_id=4DTJ45M3LSMTLKL7WPEDPBJZQQLTZNQJH5VLWMCGZHXFAULZ4ZLA, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:45.539057482 (329h17m31.001149965s) [noise=true , address=45.91.116.4:4937 , piece_id=T3U7JDJZ7NONMSTGPNQ4LAZHDRD3AJTPEVDX5KONSESNFFBAGMKQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:48:46.333169940 (329h17m31.795262414s) [noise=true , address=77.90.31.14:4661 , piece_id=3VGGQRAXFR7WWXUPXTCON5Q7COQ5P3RIMXX4W6RM5AENTTJ4URIQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: <nil> Oct 7 06:49:11.596553247 (329h17m57.05864573s) [noise=true , address=145.79.227.2:4113 , piece_id=CDZE7JG62NREH2PATTQSLBEBCPFKUDG2TKEL2RCMATSNRJOBUWTQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: read tcp 188.166.225.170:59910->145.79.227.2:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.596590589 (329h17m57.058683062s) [noise=true , address=145.79.227.14:4840 , piece_id=F72327RHJVZPDANSVTTUGMPPGI3QHWXKMGH6GYJBFHZFW5LC7YRQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.596811940 (329h17m57.058904413s) [noise=true , address=145.79.227.15:4873 , piece_id=FOI5IVBTLCOKQFZVPDIQQML42TD27DK23AWBBIHJPET7VPHMQ4YQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:48514->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.596942960 (329h17m57.059035453s) [noise=true , address=145.79.227.4:4186 , piece_id=JPND4ZFPQJJ53RDLJUAXKJKOBG32RQQHZT7V3IRDFCKE3AFSWGUA, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:41168->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.596991492 (329h17m57.059083985s) [noise=true , address=145.79.227.11:4730 , piece_id=CWXE2NNRGALUVRQGYVEXCV3R5FXGCJ7QYAGEYRJPBJ6V6G7BHZYQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.597088767 (329h17m57.05918125s) [noise=true , address=145.79.227.3:4151 , piece_id=IN4WIZE6I4TKHQB2UJYHP52T7JPG2ADU2QQ6WGDFFKBRRUYHNRKQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: read tcp 188.166.225.170:51772->145.79.227.3:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.597226089 (329h17m57.059318572s) [noise=true , address=145.79.227.14:4838 , piece_id=3R2NSXPFQW4SXVX4H6LLETXOKKIYUBQWFBLEONHGXUCK2SKA673A, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:36224->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.597240285 (329h17m57.059332758s) [noise=true , address=145.79.227.17:4971 , piece_id=52Z3SIGTDQBFFJRS2VB6M7Z3PHHUJH54UOZCP6BHC6UMXXBEUK4Q, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:33538->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.597452379 (329h17m57.059544862s) [noise=true , address=145.79.227.7:4295 , piece_id=LS57LC7KZKRSUNUYTROTOY6TNTMZBRR4772YPEBAJH4AYHSL5LAQ, seg_pos={0 0}, upload=289786]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:45286->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:49:11.597493858 (329h17m57.059586331s) [seg_pos={0 0}, upload=289786]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Oct 7 06:49:11.843701359 (329h17m57.305793842s) [upload=289786]: Done uploading segments... Oct 7 06:49:11.843733911 (329h17m57.305826384s) [upload=289786]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration:* 30.2494477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................................X....X.XXX.] duration: 48.311615752s FAIL (2025-10-07 06:48:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== MVGTHVY75ZLSY7UWAC5SQGYY3TTTW3RWTYOYSXHOEO6SZ4FKAA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705111464 (329h15m23.167203937s) [noise=true , address=77.90.31.12:4606 , piece_id=IXAIGCSWW7VY3HFK7CF7CNLNM3TASDDBWJSRTOHGEWVQAP7ADHBQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705487209 (329h15m23.167579692s) [noise=true , address=145.79.227.23:4552 , piece_id=GVG2QYK6ZLFUQ65P4X6GIEC7HAVLP7EI4EBGDX5KGFIAAMQPIZ2Q, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705497098 (329h15m23.167589571s) [noise=true , address=77.90.31.13:4629 , piece_id=NRRW6N2T6YCTRMDDB3ZSWEOLTKQI2F3GKJDEBVVF65KXE3WJHKLA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705904694 (329h15m23.167997177s) [noise=true , address=77.90.31.10:4531 , piece_id=ORCE3C4ICPSKCAGUKXEUSXIF2SEDZNRT3BUILJ4YUMP7QXLBHVCQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705959458 (329h15m23.168051932s) [noise=true , address=45.91.116.9:4270 , piece_id=FJFEUAGU7L4OKYIBKYQTHIRFVYT7CN3HIDMIE6AXEWLECZLBEETQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706211489 (329h15m23.168303962s) [noise=true , address=45.91.116.7:4043 , piece_id=CETYJJ4N5ZLRZSAAR2AH7R5OMQ7HVF5RJTNZBDUSDAKNKYXZUQGA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706330585 (329h15m23.168423068s) [noise=true , address=145.79.227.22:4513 , piece_id=NMGWKE7YVNOIXEP4D32M3YLGHKPM45V5T3GFF5R5TLASNY7N4ADA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706491422 (329h15m23.168583895s) [noise=true , address=77.90.31.17:4804 , piece_id=LOWJXG2XRC3FVDBEVS266D7E5AFHAH6XF6AGGQUYCEWYHHZECJBQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706669350 (329h15m23.168761823s) [noise=true , address=77.90.31.11:4575 , piece_id=ESE4YTFCLCX34UFK6GLUGMWDOIZJRFD5ZJAHNRSGX5RSWR2MQLEA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706792574 (329h15m23.168885078s) [noise=true , address=77.90.31.21:4890 , piece_id=BFVG3TIT6Q7NDHLVOXRTJSDZ7STT3RGBLCXAAKCB5GOHO55MPN5A, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706989410 (329h15m23.169081883s) [noise=true , address=77.90.31.16:4738 , piece_id=S3XFQKWU6B5FYUBQXLLK2ILOXHXRAYWI47MXGULEP53QQT2BZAOA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707090182 (329h15m23.169182655s) [noise=true , address=145.79.227.27:4701 , piece_id=IDEVO4UTDUO5X6DO2YQUYHHGRMEEL7A6QEIOCQZ6XMVYFJZMQQOQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707276636 (329h15m23.169369109s) [noise=true , address=45.91.116.8:4060 , piece_id=5AXATBD5ZHE3DDLE6BGXB3W7WVKRXYKM25BYR2TNU7RFGCRCJZSQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707371978 (329h15m23.169464451s) [noise=true , address=145.79.227.33:4894 , piece_id=DMSAEZYC5FQDZCDJIGD3JN7OPIAQLP6U5RGKZXRYNPZQRP5MLUUQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707549766 (329h15m23.16964225s) [noise=true , address=145.79.227.52:4454 , piece_id=NM44WSGL6ZFTCS3JZ7XGXWXBWEZBUS4BHHP4VNZRJXPBGPWDNYWA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707631533 (329h15m23.169724006s) [noise=true , address=145.79.227.26:4670 , piece_id=SUXAM2K4NBOY5JU2VHCFY3SPSYWR2LC4KSBID56RXQ6DU2OUFTVA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.708072953 (329h15m23.170165436s) [noise=true , address=145.79.227.16:4928 , piece_id=EZZSSR4MKZ2PLU7XB7KRZM47FWCJXYLYTFIL6LICB57JCE7ZF7LA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:39.225707749 (329h15m24.687800222s) [upload=289755]: Next returned nil segment Oct 7 06:46:39.225732907 (329h15m24.68782539s) [upload=289755]: Waiting for error group managing segments... Oct 7 06:46:39.615721759 (329h15m25.077814242s) [noise=true , address=45.91.116.9:4270 , piece_id=FJFEUAGU7L4OKYIBKYQTHIRFVYT7CN3HIDMIE6AXEWLECZLBEETQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.643120521 (329h15m25.105213004s) [noise=true , address=77.90.31.21:4890 , piece_id=BFVG3TIT6Q7NDHLVOXRTJSDZ7STT3RGBLCXAAKCB5GOHO55MPN5A, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.679706697 (329h15m25.14179918s) [noise=true , address=77.90.31.11:4575 , piece_id=ESE4YTFCLCX34UFK6GLUGMWDOIZJRFD5ZJAHNRSGX5RSWR2MQLEA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.682246323 (329h15m25.144338806s) [noise=true , address=77.90.31.17:4804 , piece_id=LOWJXG2XRC3FVDBEVS266D7E5AFHAH6XF6AGGQUYCEWYHHZECJBQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.682520384 (329h15m25.144612857s) [noise=true , address=45.91.116.8:4060 , piece_id=5AXATBD5ZHE3DDLE6BGXB3W7WVKRXYKM25BYR2TNU7RFGCRCJZSQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.727919902 (329h15m25.190012375s) [noise=true , address=77.90.31.16:4738 , piece_id=S3XFQKWU6B5FYUBQXLLK2ILOXHXRAYWI47MXGULEP53QQT2BZAOA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.732209110 (329h15m25.194301593s) [noise=true , address=45.91.116.7:4043 , piece_id=CETYJJ4N5ZLRZSAAR2AH7R5OMQ7HVF5RJTNZBDUSDAKNKYXZUQGA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.746507747 (329h15m25.20860022s) [noise=true , address=77.90.31.10:4531 , piece_id=ORCE3C4ICPSKCAGUKXEUSXIF2SEDZNRT3BUILJ4YUMP7QXLBHVCQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.793149551 (329h15m25.255242044s) [noise=true , address=77.90.31.12:4606 , piece_id=IXAIGCSWW7VY3HFK7CF7CNLNM3TASDDBWJSRTOHGEWVQAP7ADHBQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.989452452 (329h15m25.451544926s) [noise=true , address=45.91.116.10:4089 , piece_id=6ZMVGTHVY75ZLSY7UWAC5SQGYY3TTTW3RWTYOYSXHOEO6SZ4FKAA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:40.810206582 (329h15m26.272299066s) [noise=true , address=77.90.31.13:4629 , piece_id=NRRW6N2T6YCTRMDDB3ZSWEOLTKQI2F3GKJDEBVVF65KXE3WJHKLA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:47:36.689555151 (329h16m22.151647634s) [noise=true , address=145.79.227.26:4670 , piece_id=SUXAM2K4NBOY5JU2VHCFY3SPSYWR2LC4KSBID56RXQ6DU2OUFTVA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1Tn26F2GhEKm5HP7P7SFYjyBsRt31ueACyGzXdRYoPwEsK7jqp, address:145.79.227.26:4670): protocol: rpc: dial tcp 145.79.227.26:4670: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.689594726 (329h16m22.151687219s) [noise=true , address=145.79.227.23:4552 , piece_id=GVG2QYK6ZLFUQ65P4X6GIEC7HAVLP7EI4EBGDX5KGFIAAMQPIZ2Q, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12YmWSZZQgXm2gdZCahnyCdSzxYPAerzCfD38mWPmpTDyGYT823, address:145.79.227.23:4552): protocol: read tcp 188.166.225.170:47042->145.79.227.23:4552: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.689879608 (329h16m22.151972091s) [noise=true , address=145.79.227.52:4454 , piece_id=NM44WSGL6ZFTCS3JZ7XGXWXBWEZBUS4BHHP4VNZRJXPBGPWDNYWA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2, address:145.79.227.52:4454): protocol: read tcp 188.166.225.170:57220->145.79.227.52:4454: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690032350 (329h16m22.152124843s) [noise=true , address=145.79.227.27:4701 , piece_id=IDEVO4UTDUO5X6DO2YQUYHHGRMEEL7A6QEIOCQZ6XMVYFJZMQQOQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1weCZWAtZRRYdeuPE8TjnZB5fZVD4gz4qW6TxyyNz33wVfW9hQ, address:145.79.227.27:4701): protocol: read tcp 188.166.225.170:56690->145.79.227.27:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690084148 (329h16m22.152176641s) [noise=true , address=145.79.227.16:4928 , piece_id=EZZSSR4MKZ2PLU7XB7KRZM47FWCJXYLYTFIL6LICB57JCE7ZF7LA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1fKSEPa36EtKVhzFZ2tCsQVi62RynY5Bwup1kQE17UB1bswsHg, address:145.79.227.16:4928): protocol: read tcp 188.166.225.170:49436->145.79.227.16:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690243111 (329h16m22.152335594s) [noise=true , address=145.79.227.33:4894 , piece_id=DMSAEZYC5FQDZCDJIGD3JN7OPIAQLP6U5RGKZXRYNPZQRP5MLUUQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44, address:145.79.227.33:4894): protocol: read tcp 188.166.225.170:41880->145.79.227.33:4894: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690289158 (329h16m22.152381651s) [noise=true , address=145.79.227.22:4513 , piece_id=NMGWKE7YVNOIXEP4D32M3YLGHKPM45V5T3GFF5R5TLASNY7N4ADA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: read tcp 188.166.225.170:34444->145.79.227.22:4513: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690322742 (329h16m22.152415225s) [seg_pos={0 0}, upload=289755]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Oct 7 06:47:36.934316798 (329h16m22.396409271s) [upload=289755]: Done uploading segments... Oct 7 06:47:36.934355181 (329h16m22.396447664s) [upload=289755]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 48.311635609s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)