Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
31.234355946s
FAIL (2025-10-20 10:42:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 31.234381124s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
30.560394717s
FAIL (2025-10-20 10:41:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 30.560426888s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
32.934519874s
FAIL (2025-10-20 10:41:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 32.934554639s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
25.107591402s
FAIL (2025-10-20 10:40:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 25.107629906s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
37.74393446s
FAIL (2025-10-20 10:39:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 37.743964917s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
27.185814213s
FAIL (2025-10-20 10:37:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 27.185869488s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
30.449680463s
FAIL (2025-10-20 10:37:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 30.449700712s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
21.237222654s
FAIL (2025-10-20 10:37:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 21.237256328s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
10.909235285s
FAIL (2025-10-20 10:36:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 10.909270011s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
45.473650897s
FAIL (2025-10-20 10:35:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 45.473696774s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
38.967775918s
FAIL (2025-10-20 10:34:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 38.967813158s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
13.13182075s
FAIL (2025-10-20 10:33:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 13.131856289s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
44.735887886s
FAIL (2025-10-20 10:33:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 44.735922633s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
35.532340694s
FAIL (2025-10-20 10:31:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 35.532374087s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
21.443796478s
FAIL (2025-10-20 10:30:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 21.443831265s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
15.021930229s
FAIL (2025-10-20 10:29:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 15.021960068s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
13.145884384s
FAIL (2025-10-20 10:28:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 13.145921734s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
14.241442346s
FAIL (2025-10-20 10:28:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 14.241471771s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
36.792705948s
FAIL (2025-10-20 10:27:58 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 36.792742949s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
15.291784534s
FAIL (2025-10-20 10:27:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 15.291807247s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
30.436599851s
FAIL (2025-10-20 10:26:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 30.436624067s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
40.825835578s
FAIL (2025-10-20 10:26:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 40.825867036s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
12.782443761s
FAIL (2025-10-20 10:25:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 12.782702763s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
18.794101968s
FAIL (2025-10-20 10:24:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 18.794141834s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
29.875447903s
FAIL (2025-10-20 10:23:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 29.875479943s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
33.112702292s
FAIL (2025-10-20 10:22:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 33.112727119s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
39.224135439s
FAIL (2025-10-20 10:21:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 39.224196014s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
11.947095755s
FAIL (2025-10-20 10:20:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 11.947143086s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
41.084600778s
FAIL (2025-10-20 10:19:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 41.084634201s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
15.763581172s
FAIL (2025-10-20 10:18:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 15.763608695s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
46.675904978s
FAIL (2025-10-20 10:18:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 46.675940377s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
47.61986352s
FAIL (2025-10-20 10:17:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 47.619932552s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.004235374s
FAIL (2025-10-20 10:16:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
upload=567220]: Uploading piece...
Oct 20 10:15:08.876561476 (644h43m54.338653949s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.876937341 (644h43m54.339029824s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877402678 (644h43m54.339495161s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.877772892 (644h43m54.339865375s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879077116 (644h43m54.341169599s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.879972922 (644h43m54.342065405s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.881411190 (644h43m54.343503674s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882030791 (644h43m54.344123274s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.882934792 (644h43m54.345027285s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:08.883965845 (644h43m54.346058338s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Uploading piece...
Oct 20 10:15:09.716622616 (644h43m55.178715099s) [upload=567220]: Next returned nil segment
Oct 20 10:15:09.716644438 (644h43m55.178736921s) [upload=567220]: Waiting for error group managing segments...
Oct 20 10:15:10.085029662 (644h43m55.547122145s) [noise=true , address=81.171.14.70:4045 , piece_id=7ZORELQQD2L4LQ7J3WP47NMSX6INH6JNNQHJWBMOR2EC5YGE6IZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.105175333 (644h43m55.567267816s) [noise=true , address=81.171.14.70:4030 , piece_id=7CH4AG3BUNMOXHH4EEIQBCUAG3P7K4ZED5P4JBTTMI424EFQZRHQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.110535049 (644h43m55.572627532s) [noise=true , address=81.171.14.70:4035 , piece_id=WP6XUKMOYNB7CMWPSAXC6QJXR6FUMSBENYDMSBL3LB36C2MVRF7A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.116437968 (644h43m55.578530451s) [noise=true , address=81.171.14.70:4042 , piece_id=Y5VPEIHQZQ3WSMV35O7ECFD7X7XLCZYGCQWCMUQ6PICKRWRXSXKQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.118951796 (644h43m55.581044279s) [noise=true , address=81.171.14.70:4038 , piece_id=FPBOIKH3QGAHYRBXNXU6DVFWJPKBLF54LGS7FDT4PPMX2WBZBVGA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.136988531 (644h43m55.599081014s) [noise=true , address=89.149.201.147:4011 , piece_id=H6HZLRGFMVECQS7BZ6XBPRHL5VOHDK3Y7JW2RL6GELYYN7VR4CDA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.167427673 (644h43m55.629520156s) [noise=true , address=89.149.201.147:4007 , piece_id=JSZNIVSEXPKQSRRTAYANKCXXB2E5XLUJUYLAKM5LCJ4OZGL7CNCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.170999944 (644h43m55.633092417s) [noise=true , address=89.149.201.147:4010 , piece_id=XK2BJ7VZDSN4NUOAUHDPDRGODEYVORZBFSLE7VBVFKEM46V4B2GQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.174261996 (644h43m55.636354479s) [noise=true , address=89.149.201.147:4021 , piece_id=TZGL3DNXOQH6L3HW6RQEQ6NWDZ2UNRWXVDEY7WKWVFGO33RQJNUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.179813286 (644h43m55.641905779s) [noise=true , address=89.149.201.147:4012 , piece_id=G4YZ7YNNUHSL2IL6VODIHBG6OIGWNJ3LAWJLPBB254BNZVJZBAUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.209318801 (644h43m55.671411284s) [noise=true , address=81.171.14.70:4041 , piece_id=GJ27VFK7N5VSWZ6ICDPNZYFZGC2YDVP2L5AUHFUDIEVRM5A7D6BA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.210997598 (644h43m55.673090081s) [noise=true , address=89.149.201.147:4014 , piece_id=HNC7LNCVE3YIOD5CBLZO2QFBMALIG3UHJX4DRLEXRJDC4DLSZOCQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.221536877 (644h43m55.68362937s) [noise=true , address=81.171.14.70:4026 , piece_id=PI5I73VL625BN6NKTTMN3JN4YSLHDUNEU54IBS43TBFCLCOCX6NQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.307797863 (644h43m55.769890347s) [noise=true , address=89.149.201.147:4020 , piece_id=LLCJR5CHHW7L4JN5QWOXJW523RT326QLHI7VXAJRMYX5J2M6CE6Q, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.398895680 (644h43m55.860988154s) [noise=true , address=81.171.14.70:4046 , piece_id=UUW7CGNTMYDYVPN5J55ZXBGDAFNPUMJLANV5CGC2IODO7CULATUA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.399352270 (644h43m55.861444753s) [noise=true , address=81.171.14.70:4031 , piece_id=B5QEJOJUIDHNED6Q7HDCJOD5ON3OHJQLPO5A2JYOTPT2G56T7NLA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464057109 (644h43m55.926149582s) [noise=true , address=89.149.201.147:4015 , piece_id=KLIKA3LIRKUSKJ6RTWV3HLYYETKZ5N6QM2JHQYTMPEMVBCGLDHNA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.464709462 (644h43m55.926801955s) [noise=true , address=89.149.201.147:4006 , piece_id=IHIL3BFO5VUAH27SDBMLVWD3GB2V3BXPQPDH762VRRKKOG3RAZSA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.754481682 (644h43m56.216574165s) [noise=true , address=185.167.204.205:4018 , piece_id=Z2KS6OS4SB5IUBSPBUTYB7I52AA3OBHETLIQCT6KU5IUJDGU7L5A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.763916587 (644h43m56.22600907s) [noise=true , address=185.167.204.205:4035 , piece_id=6WTP7EVRYS7HACJUA65ZNN5U4QAEJFF7R335JAAZA52U2E2IIFZA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.792657716 (644h43m56.254750189s) [noise=true , address=185.167.204.206:4056 , piece_id=3VO4EP4S3VE673E72N63GHVALURG2TKQU5N5U4L7XXD2VJVDHXHA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.801524770 (644h43m56.263617263s) [noise=true , address=89.149.201.147:4009 , piece_id=MN5NCR5C52QZ4UCSVO7SJESRTCJNJTEARZRRZWZM4R6NCDY3JS3A, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.850271713 (644h43m56.312364196s) [noise=true , address=81.171.14.70:4025 , piece_id=TLK3IEMTYNBHQQTU3HZBCHQM2JCHPDBJ2TOJMFLKGHN3QR43Y6UQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:10.900170760 (644h43m56.362263243s) [noise=true , address=185.167.204.206:4063 , piece_id=P37UG4LCWIDCF3WD3SA35YTAX5XJYJB3ICVRTPUOJWLYPAIYIYWQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:11.323542798 (644h43m56.785635291s) [noise=true , address=185.167.204.206:4051 , piece_id=3DHWQLDJ3ZOXDQCQL6FAUFINJUZLDMZAJAJOK7OZNESSXCQKBLGQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220458823 (644h43m57.682551306s) [noise=true , address=185.167.204.205:4009 , piece_id=CO37EZCGGWTOYY264IM24QCAN6EVV5YMI3VDFAYJXESBCY4RXRVQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: <nil>
Oct 20 10:15:12.220485533 (644h43m57.682578016s) [seg_pos={0 0}, upload=567220]: Segment reached optimal threshold of 26 pieces.
Oct 20 10:15:12.220604760 (644h43m57.682697243s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220613657 (644h43m57.68270614s) [noise=true , address=185.167.204.205:4014 , piece_id=DUXCQOUW36CUW5CIBRGYE2FI6PXECZEJZH57ZU356P4TAQ2EWTQQ, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220739327 (644h43m57.68283181s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR): protocol: expected piece hash; context canceled; EOF
Oct 20 10:15:12.220748594 (644h43m57.682841077s) [noise=true , address=185.167.204.205:4031 , piece_id=RMXZAKE43FCCL2IMQZ3TV6AGLGZKVFUG4RR3BRDRYYNBJB3F7NQA, seg_pos={0 0}, upload=567220]: Long tail cancellation due to optimal threshold
Oct 20 10:15:12.220770757 (644h43m57.68286324s) [seg_pos={0 0}, upload=567220]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 20 10:15:12.220854255 (644h43m57.682946738s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 20 10:15:12.220861869 (644h43m57.682954342s) [upload=567220]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 20 10:15:12.511361023 (644h43m57.973453516s) [upload=567220]: Done uploading segments...
Oct 20 10:15:12.511387544 (644h43m57.973480027s) [upload=567220]: Upload finished. err: <nil>
Duration: 1m0.004286541s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
6.037366027s
FAIL (2025-10-19 19:21:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.037421082s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.215089472s
FAIL (2025-10-19 19:21:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.215119969s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.167752505s
FAIL (2025-10-19 19:21:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.167790105s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.065765451s
FAIL (2025-10-19 19:20:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.065871705s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.944197428s
FAIL (2025-10-19 19:19:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 5.944248656s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.064003408s
FAIL (2025-10-19 19:18:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.064080996s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.862318976s
FAIL (2025-10-19 19:17:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 5.862352789s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
6.083600574s
FAIL (2025-10-19 19:17:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.083644638s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.29968823s
FAIL (2025-10-19 19:16:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.299726061s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.023288155s
FAIL (2025-10-19 19:15:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.023350614s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
6.216451726s
FAIL (2025-10-19 19:14:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.216483645s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
6.071163772s
FAIL (2025-10-19 19:13:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.071197737s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
6.2613041s
FAIL (2025-10-19 19:12:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.261336381s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
6.313043492s
FAIL (2025-10-19 19:11:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.313074303s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.974401119s
FAIL (2025-10-19 19:10:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 5.974448088s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
6.414520281s
FAIL (2025-10-19 19:10:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 6.414567903s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.981805382s
FAIL (2025-10-19 19:09:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
ise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.036586942 (629h4m18.498679425s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037250595 (629h4m18.499343068s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.037767289 (629h4m18.499859772s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038222085 (629h4m18.500314558s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038656051 (629h4m18.500748524s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.038959800 (629h4m18.501052273s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039271113 (629h4m18.501363586s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039597424 (629h4m18.501689897s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.039918586 (629h4m18.502011069s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.041196769 (629h4m18.503289242s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.042526161 (629h4m18.504618674s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.043780159 (629h4m18.505872642s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.044954816 (629h4m18.507047309s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.046298554 (629h4m18.508391037s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:33.047496396 (629h4m18.509588879s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Uploading piece...
Oct 19 18:35:34.709975582 (629h4m20.172068055s) [upload=553457]: Next returned nil segment
Oct 19 18:35:34.709997383 (629h4m20.172089856s) [upload=553457]: Waiting for error group managing segments...
Oct 19 18:35:34.985328199 (629h4m20.447420672s) [noise=true , address=145.79.227.4:4187 , piece_id=WHEUJZ7CJGBHGYVJPST5FGZXOB5S2QITCRQJLXIMFZA4XB6XVXNA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.007306921 (629h4m20.469399404s) [noise=true , address=145.79.227.11:4732 , piece_id=CZTMB3BB637GDSLPNJJCFZYE74ECGUJJT7I3YFGETOGT6VO4TZUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.188007943 (629h4m20.650100416s) [noise=true , address=145.79.227.12:4765 , piece_id=VUKLHA6FTH4O3WTZ7ZGZJKLRDCGKA4QIIXR2W5URHOIMP4Z7TDFQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.233956739 (629h4m20.696049222s) [noise=true , address=145.79.227.17:4970 , piece_id=WCCTQFETSSKDSEXCP34HICB7ZQK4TKDZEOICNCC67ETJ53SDBWZA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.309468910 (629h4m20.771561393s) [noise=true , address=145.79.227.6:4257 , piece_id=QEUMAFNWWPQODAPRNMU6VTPQD32GEYY4AMY4KSO4HCXIL6SXKUMA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.315497268 (629h4m20.777589752s) [noise=true , address=145.79.227.16:4911 , piece_id=SICZRCPW2Z52EUDZIPNB36LIG3F62E5YNYVZZRWXOA5C67C5KTNQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.380650454 (629h4m20.842742947s) [noise=true , address=145.79.227.18:4006 , piece_id=JZGDLTO3TEOG7MY34XELOMAOIFYNBLMD2GD5EJQ43RPENYDPE3BA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:35:35.451168493 (629h4m20.913260977s) [noise=true , address=145.79.227.14:4839 , piece_id=ISIC7NO7BUXHVBEBQA7I2B6NC5ZQG7UPJQZO4L3GC5ZWRCJMBYHA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:00.917983945 (629h4m46.380076428s) [noise=true , address=45.91.116.8:4047 , piece_id=2AZYJ7T7UT4HGWY6REL6LJNHS3S2S7HYNZYN2TYGQD2XNNIQOFHQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: <nil>
Oct 19 18:36:02.057330282 (629h4m47.519422785s) [noise=true , address=77.90.31.19:4842 , piece_id=SKYUQDAO37UAWXMDSU7WQK7SD63AICZZWXILB3C77Q44IEO4LFPA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47382->77.90.31.19:4842: use of closed network connection
Oct 19 18:36:02.057639892 (629h4m47.519732385s) [noise=true , address=45.91.116.14:4189 , piece_id=BYHB66SJJZVHCEALWZYWITQX63W7GWDRYJWUQ6CW3KUQEXJ7YTAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48326->45.91.116.14:4189: use of closed network connection
Oct 19 18:36:02.057741676 (629h4m47.519834159s) [noise=true , address=45.91.116.5:4299 , piece_id=UU73PZDZVC626L2OVOU4CE4NUQK6WXYCB5YWHZDKLZBTWF7BOC6A, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44290->45.91.116.5:4299: use of closed network connection
Oct 19 18:36:02.057910437 (629h4m47.52000292s) [noise=true , address=45.91.116.11:4119 , piece_id=CYZATFHUSRFGRKLXBD6J4HSRZIFN7OPDKWQDXLPOYUSGD3VYQSUQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58982->45.91.116.11:4119: use of closed network connection
Oct 19 18:36:02.057936457 (629h4m47.52002894s) [noise=true , address=77.90.31.15:4698 , piece_id=SSG44QWSHC654P5BL4J3L7ESSJTVKEXOC5K34YNB2D5GFZ67E2WQ, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49834->77.90.31.15:4698: use of closed network connection
Oct 19 18:36:02.058127550 (629h4m47.520220043s) [noise=true , address=77.90.31.13:4628 , piece_id=MZKHBG7X3XKCAXHNYLXPL4HXE7QNXLKGINUTKO5TOOX4KOXCOKKA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:35616->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 18:36:02.058183247 (629h4m47.52027573s) [noise=true , address=77.90.31.17:4772 , piece_id=NLUQVEQCPXYCKSIA4C74255DKCNPC7KLRKH2NVDTGX5E4DD6ZLAA, seg_pos={0 0}, upload=553457]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42794->77.90.31.17:4772: use of closed network connection
Oct 19 18:36:02.058233211 (629h4m47.520325695s) [seg_pos={0 0}, upload=553457]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 19 18:36:02.314508332 (629h4m47.776600825s) [upload=553457]: Done uploading segments...
Oct 19 18:36:02.314566082 (629h4m47.776658565s) [upload=553457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 5.981839586s
core
uplink
us-select-1
|