Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration:
4.863015862s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:45:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.863042401s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration:
4.875230903s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:44:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.875256292s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [................XXX.........................................] duration:
4.881513665s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:43:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.881539262s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.................XXX........................................] duration:
5.033664264s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:42:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.033696417s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..................XXX.......................................] duration:
4.906844932s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:41:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.906877833s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...................XXX......................................] duration:
4.85301113s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:41:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.853049402s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [....................XXX.....................................] duration:
4.886897826s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:40:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.886942562s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.....................XXX....................................] duration:
4.923789448s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:39:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.923818744s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [......................XXX...................................] duration:
4.861466377s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:39:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.861496616s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration:
4.898002943s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:38:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.898033842s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [........................XXX.................................] duration:
4.898509041s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:37:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.898541224s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.........................XXX................................] duration:
4.949035194s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:37:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.949062636s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration:
4.849535346s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:36:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.849564712s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration:
4.859143824s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:36:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.859181176s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [............................XXX.............................] duration:
4.873895521s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:35:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.873928124s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.............................XXX............................] duration:
5.17855892s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:34:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.178592293s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration:
4.902204708s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:33:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.902235427s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration:
4.848585838s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:32:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.848617269s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [................................XXX.........................] duration:
5.255242061s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:31:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.25527296s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.................................XXX........................] duration:
4.87145194s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:30:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.871506405s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration:
4.881727297s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:30:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.881754499s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...................................XXX......................] duration:
5.269702707s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:29:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.269731111s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration:
5.192009072s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:28:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.192112628s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration:
5.301366159s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:28:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.301397808s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [......................................XXX...................] duration:
4.855824459s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:27:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.855988793s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration:
5.269845337s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:26:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.269875824s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [........................................XXX.................] duration:
4.87706057s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:25:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.877089024s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.........................................XXX................] duration:
4.878942484s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:24:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.879049217s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration:
5.348105923s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:23:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.348131774s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration:
4.866242109s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:22:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.86627946s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [............................................XXX.............] duration:
4.842301246s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:21:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.842333077s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.............................................XXX............] duration:
5.271714668s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:20:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.271743153s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration:
4.837508262s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:19:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.837534895s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration:
5.278883377s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:18:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.278908855s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [................................................XXX.........] duration:
4.816830386s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:18:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.816857326s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.................................................XXX........] duration:
4.843315453s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:17:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.843349309s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration:
5.257641942s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:16:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.257670455s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...................................................XXX......] duration:
4.825336928s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:15:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.825363317s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration:
5.185182453s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:15:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.185207139s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration:
4.848077542s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:14:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.848110475s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [......................................................XXX...] duration:
4.930598037s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:14:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 4.930625008s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration:
5.195947189s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:13:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 5.195978049s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [........................................................XXX.] duration:
58.408869858s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:13:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 58.408888393s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.175624953s trace:
https://jaeger.dev.storj.io/trace/45bd9e2340ba34ac
FAIL (2025-11-12 19:12:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:06.578920173 (83h7m5.130390634s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Uploading piece...
Nov 12 19:11:07.839593682 (83h7m6.391064143s) [upload=73166]: Next returned nil segment
Nov 12 19:11:07.839609292 (83h7m6.391079753s) [upload=73166]: Waiting for error group managing segments...
Nov 12 19:11:08.407010437 (83h7m6.958480898s) [noise=true , address=185.167.204.205:4029 , piece_id=UIZC3MXQMVNDCKZJUPRKJZZFB3X4LPDZZMAV52OZU46QFQJKE5QA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.420417297 (83h7m6.971887758s) [noise=true , address=185.167.204.206:4060 , piece_id=EWPYOIFQ4SJKCW5YBNSOCT275DMPD2AX3VNZAWH4R4XAMGBTXZIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.421074849 (83h7m6.97254531s) [noise=true , address=185.167.204.206:4059 , piece_id=XIOMFA7CGVX222E6EUBI6FXZ3U7MARY7Z6KS5WYCNHH3FLFZ6BDQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.430081610 (83h7m6.981552071s) [noise=true , address=89.149.201.147:4001 , piece_id=YJMPTIK2K2QAW3IFKGNELC34UKHF6IKRWTMLKQOGYJ4UQDLHXJYQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.440139683 (83h7m6.991610134s) [noise=true , address=89.149.201.147:4009 , piece_id=5IFGQ4MRD3DMYB6T7MZMCO4H5OO5ORNEGT3NHKUO6GWFODFHW72A, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.446882252 (83h7m6.998352723s) [noise=true , address=89.149.201.147:4023 , piece_id=TQXKWXUTM67HXI3WVECCUP7LYNFCYTJWJSLB4C22QECJD7JATBJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.452158048 (83h7m7.003628509s) [noise=true , address=185.167.204.205:4023 , piece_id=LVJTWTPMRFGIHJSUKSHXDD7Y5M7LRT3RALKEWY3AWICDI4DWFUJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.455220790 (83h7m7.006691241s) [noise=true , address=89.149.201.147:4018 , piece_id=UZTDCCR437NYVJKXF4OE2HAWK254645GZLUWBJL3WG4QAATUOIZA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.461101409 (83h7m7.01257187s) [noise=true , address=185.167.204.206:4038 , piece_id=WZGDCB4HEIH25IHB5BKVQ47O7G4ROGPLRURT2F3VLFP5YHWY4XTA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.468258387 (83h7m7.019728848s) [noise=true , address=185.167.204.205:4003 , piece_id=ZGXXWXQ2TQJ3EUFUAIBYPJLICHUXEW4KORPUKZNCQFDYAWHCG6PA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.470585378 (83h7m7.02205583s) [noise=true , address=185.167.204.206:4050 , piece_id=UOCVSBIZLDW3ERLKRZFU5Q74MIOW7R3EAIDXNVW4I6REYICFXNOA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.473600199 (83h7m7.02507066s) [noise=true , address=89.149.201.147:4020 , piece_id=RPBWVH3T6QO2VM5QCT55UBR54I3T3G5FX4OLMXL5XLGSVWE4MFHA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.481643796 (83h7m7.033114257s) [noise=true , address=89.149.201.147:4005 , piece_id=G2EMQTB5RQXQE555YL2CY7AKZ6NESS3ICOIZHJS6FFIKGXDAYJ4Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.492150463 (83h7m7.043620924s) [noise=true , address=185.167.204.205:4009 , piece_id=ZHLVEKZTIIMINBGQWYDE3USJAGLMW45DNYLFW5HFIJUZKRCVP4UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.496844992 (83h7m7.048315443s) [noise=true , address=185.167.204.205:4011 , piece_id=FMQIJMV6ZMWICCDVFKXCCRFOSZOKM63OWUGHITONANIKYOM7J4IA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.502936351 (83h7m7.054406812s) [noise=true , address=185.167.204.205:4001 , piece_id=P6ZVMPNCSXN6HDXB75UJMARP4QFE7B5KAQOFYIMG4SLFRYVYGMJA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.506297802 (83h7m7.057768263s) [noise=true , address=185.167.204.205:4015 , piece_id=K5O7PVBSBZSH73AB6CXRK36OUGZHKVASJ6FVEX4KLMOE3XQSGONQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.513261863 (83h7m7.064732324s) [noise=true , address=89.149.201.147:4010 , piece_id=MVC4WF3IBUQQGRVIFJYAPDV5HY3EWFYQLLMV7YPDTQ3VM5DQNUXQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.521414257 (83h7m7.072884718s) [noise=true , address=89.149.201.147:4016 , piece_id=U7WDPLCA2KFPNAFTZVI6VRA32BR4SALBZKTA3I5ZTDC3ROP6UFMA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:11:08.566177936 (83h7m7.117648397s) [noise=true , address=185.167.204.205:4020 , piece_id=PIKSGY7MSEYSMYMQO7D32DKHKYED5KAMUE3VOJG5IUUHA5BHOG2Q, seg_pos={0 0}, upload=73166]: Done uploading piece. err: <nil>
Nov 12 19:12:05.878997613 (83h8m4.430468064s) [noise=true , address=81.171.14.70:4032 , piece_id=QKSKUPD4JHK7FCK2OYFBY2SHGD5X2452HW32XXGX5IP7MKPWATAA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41762->81.171.14.70:4032: use of closed network connection
Nov 12 19:12:05.879230216 (83h8m4.430700677s) [noise=true , address=81.171.14.70:4033 , piece_id=UJNKIYWFZ2AFPCF6XND243VYNTEJ2WAHASLGWCF5LSCUSAT6ZQKA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51970->81.171.14.70:4033: use of closed network connection
Nov 12 19:12:05.879559022 (83h8m4.431029483s) [noise=true , address=81.171.14.70:4029 , piece_id=UK4Q2AF5AWQOPYAIZSFCJVYNKPEFCIW74T45FQHVCRSQPARUZHBA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43980->81.171.14.70:4029: use of closed network connection
Nov 12 19:12:05.879857471 (83h8m4.431327962s) [noise=true , address=81.171.14.70:4040 , piece_id=GJGAYFJCLOMV4M42SKLAHBZLEFCFP4WQ4HBI36VWRWOHSERMKXYA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 188.166.225.170:41536->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:12:05.879900713 (83h8m4.431371164s) [noise=true , address=81.171.14.70:4037 , piece_id=TVFZ27I67CYRXDIAVAIA2KYN5EQ75LHLAO42S7FB5HUKKBBDD7UA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60340->81.171.14.70:4037: use of closed network connection
Nov 12 19:12:05.880009840 (83h8m4.431480301s) [noise=true , address=81.171.14.70:4041 , piece_id=VFXIWHVJI6PSSUD522IQGKVOEHRRCGJLGMXSUTDHZ2TRPWLHMFVA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33734->81.171.14.70:4041: use of closed network connection
Nov 12 19:12:05.880095664 (83h8m4.431566115s) [noise=true , address=81.171.14.70:4035 , piece_id=KEMS4YFGLTJIOIUH4VPOYGNHIRBWGM5RRW63C2KNGMETVTGJFTIA, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47454->81.171.14.70:4035: use of closed network connection
Nov 12 19:12:05.880219970 (83h8m4.431690432s) [noise=true , address=81.171.14.70:4046 , piece_id=VLXRG3KN3JFFGMRWDSMKWSOEOTW45T2FJX7E5PU6XGTU63M3XLJQ, seg_pos={0 0}, upload=73166]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34008->81.171.14.70:4046: use of closed network connection
Nov 12 19:12:05.880265837 (83h8m4.431736299s) [seg_pos={0 0}, upload=73166]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Nov 12 19:12:06.052394579 (83h8m4.60386504s) [upload=73166]: Done uploading segments...
Nov 12 19:12:06.052446328 (83h8m4.603916789s) [upload=73166]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 1m0.175643188s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
1m0.177912463s trace:
https://jaeger.dev.storj.io/trace/1ff69634cecc8756
FAIL (2025-11-12 19:11:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 15)
======LOGS======
_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.030976456 (83h6m6.582446917s) [noise=true , address=185.167.204.205:4014 , piece_id=PS5E2WQT3SBJQMLX3EBBMQVQITIKAOUUJ3FG5VSS3TLXMKWTOMXA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.051397202 (83h6m6.602867663s) [noise=true , address=185.167.204.206:4059 , piece_id=J2N7WQBUCK3JARE46QKL7DZA2OMQNFAVGGJOOYU2JHOT7P3N5BLQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.066642101 (83h6m6.618112562s) [noise=true , address=185.167.204.205:4009 , piece_id=MRJMVX2L7X6YBIZKOZ2AODPQEXVYTWDIVBB5PJF3O4NPWGIRB7XA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.070778878 (83h6m6.622249339s) [noise=true , address=89.149.201.147:4018 , piece_id=BVM5B7UKGNZYDMPEHVDNSTPCE7VNJKVGKIBIBMOKHWH45LA4TZXQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.084307290 (83h6m6.635777751s) [noise=true , address=185.167.204.205:4002 , piece_id=ONUOMH7DCEM3UHVN5GQHTAO6Q42676O45C5H563T7JQL6X5X2I2A, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.099002642 (83h6m6.650473103s) [noise=true , address=185.167.204.205:4026 , piece_id=A5PDZUHCPWB522HHUC2CKOZZPPOPHWKJ5MVPTUCEEAPIO6BPKWYA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.113197142 (83h6m6.664667603s) [noise=true , address=185.167.204.206:4046 , piece_id=3BIIX53PSY2K33QPFCNZA63P2WJ73245VRK2HBDJHVBEGMDFMRMA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.135512435 (83h6m6.686982897s) [noise=true , address=185.167.204.206:4047 , piece_id=QEJ2DUDMQ37ER7I73RZU62XYNPG5BZGM6STQRUOJRWZO7LA4PWVQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:08.138269044 (83h6m6.689739505s) [noise=true , address=185.167.204.206:4064 , piece_id=IU4C6Y2JT4SBV74IU7WTYZ5SJOHJASTMDCAJ3XEJRD66CRHZFECA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:10:46.674568400 (83h6m45.226038861s) [noise=true , address=81.171.14.70:4029 , piece_id=LJEZVP3CH3TDJMAEBSLCLE55MH22PU357OVX2GWVFA3WRP6NG4OQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: <nil>
Nov 12 19:11:05.480927455 (83h7m4.032397916s) [noise=true , address=81.171.14.70:4030 , piece_id=SNVQ362LY2NZBBHCLNV7W5V5ELRPJ6RJFIEUIBIOK3HMBWZK5SMA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:59220->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:59220->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59220->81.171.14.70:4030: use of closed network connection
Nov 12 19:11:05.481133007 (83h7m4.032603458s) [noise=true , address=81.171.14.70:4045 , piece_id=WDD3AWH2NEIA7SJQSCARDTVGO6ZHEVHCFX4ALHJTJGNML2RW5I5Q, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:59256->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:59256->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59256->81.171.14.70:4045: use of closed network connection
Nov 12 19:11:05.481215754 (83h7m4.032686215s) [noise=true , address=81.171.14.70:4025 , piece_id=D4KSB4X6LO7DVLBDB5QIZNOX73WT4XP6CMLFDKV264F4FRVC74GQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: write tcp 188.166.225.170:55484->81.171.14.70:4025: use of closed network connection; write tcp 188.166.225.170:55484->81.171.14.70:4025: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55484->81.171.14.70:4025: use of closed network connection
Nov 12 19:11:05.481283182 (83h7m4.032753643s) [noise=true , address=81.171.14.70:4036 , piece_id=JRZEBZRAPGWYADNEZMTOBW7GLRUZHBVWIZJGOAO5GUUS3HMILCVQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:37774->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:37774->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37774->81.171.14.70:4036: use of closed network connection
Nov 12 19:11:05.481311987 (83h7m4.032782448s) [noise=true , address=81.171.14.70:4027 , piece_id=QRDWMLPQGAJNTIIGNXPCWOVCAI3CLUVERVJOKC4NRJT6SQMDZ6QQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Nov 12 19:11:05.481515936 (83h7m4.032986397s) [noise=true , address=81.171.14.70:4044 , piece_id=TJKFN3P6UQI7YE6JQ6RV6TZWAGAF7RPBIO7FVXJHRN2WPE7OGLXA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: write tcp 188.166.225.170:55442->81.171.14.70:4044: use of closed network connection; write tcp 188.166.225.170:55442->81.171.14.70:4044: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55442->81.171.14.70:4044: use of closed network connection
Nov 12 19:11:05.481658988 (83h7m4.033129449s) [noise=true , address=81.171.14.70:4041 , piece_id=ZGYDE7B7TBYHWI6QBRZPTW6YZPHMA2JCVUVWPDIKQZTUWDFZP4TQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:33686->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:33686->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33686->81.171.14.70:4041: use of closed network connection
Nov 12 19:11:05.481769047 (83h7m4.033239499s) [noise=true , address=81.171.14.70:4047 , piece_id=Q573YNZOOHASMOINDVZYD53DKD57WH4RNLKNSB4WDYQ7ZXWNY5YQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:60650->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:60650->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60650->81.171.14.70:4047: use of closed network connection
Nov 12 19:11:05.481994638 (83h7m4.033465099s) [noise=true , address=81.171.14.70:4031 , piece_id=OJOQQPFRGZY42Z5HIBSPOLDUVI7CHIMZCJL5Z57QSTK4Z4FNHPRQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:51442->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:51442->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->81.171.14.70:4031: use of closed network connection
Nov 12 19:11:05.482204507 (83h7m4.033674968s) [noise=true , address=81.171.14.70:4043 , piece_id=33IJEEVPKKM55ADHSNPQHCRPIDX7DZGRWEJJVXMNUTYLLD4P2QUA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: read tcp 188.166.225.170:56256->81.171.14.70:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:11:05.482242509 (83h7m4.033712971s) [noise=true , address=81.171.14.70:4035 , piece_id=C4KT3UHZZ53FG3KDSR2D7S3TEPYO46VARPLUKTYZLTSWOL2MCUAQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:40316->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:40316->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40316->81.171.14.70:4035: use of closed network connection
Nov 12 19:11:05.482388457 (83h7m4.033858918s) [noise=true , address=81.171.14.70:4033 , piece_id=OQ2MPEQKN46U6WOR4YTHPQ6E2LRKHQOYRVT7YFC2L5UK7XB3N2UA, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:32838->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:11:05.482525548 (83h7m4.033996009s) [noise=true , address=81.171.14.70:4042 , piece_id=YET55Q2J23M3YXZXKMEL7VWBKVPNPFBGXBTW6HWJ24TFTMW5NXBQ, seg_pos={0 0}, upload=73150]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:46418->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:11:05.482615280 (83h7m4.034085741s) [seg_pos={0 0}, upload=73150]: Done waiting for segment. successful: 15 optimal: 26 errs: <nil>
Nov 12 19:11:05.656049376 (83h7m4.207519847s) [upload=73150]: Done uploading segments...
Nov 12 19:11:05.656100923 (83h7m4.207571385s) [upload=73150]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 15)
Duration: 1m0.177944244s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.199888826s trace:
https://jaeger.dev.storj.io/trace/7d5573af359cedf0
FAIL (2025-11-12 19:09:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 13)
======LOGS======
BHVSKAF4DLKTLGBVORQG24VTJ7ANQKWKX35QAM4ZON23PZFVR5Q, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:01.920410423 (83h5m0.471880884s) [noise=true , address=185.167.204.206:4068 , piece_id=VPBDZJDCRPAOKGCPTKFXCMJGTXCNALB2YBJOU2E7VISZI4HGILYQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:01.964034734 (83h5m0.515505205s) [noise=true , address=89.149.201.147:4019 , piece_id=2L5NVBIIBNEKP2W53WU2ZLFRX3HVEU6AVSUF7ORXWB2GAXXAOWUQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:01.966593215 (83h5m0.518063686s) [noise=true , address=89.149.201.147:4024 , piece_id=64LF2WFJHNDK5KWASEK62CICZZEULGPKWWKSD3XCM5T6VE6VQU6Q, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:01.967500744 (83h5m0.518971205s) [noise=true , address=89.149.201.147:4023 , piece_id=FNQTVSIVDWBXQOS3OYECYYPC5OQLC5Z7PFVRXSJUPJ4K6VDJRXMA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:01.972807499 (83h5m0.52427797s) [noise=true , address=89.149.201.147:4018 , piece_id=OXB3GFFQU6EOZTSVPTJG7UZRJG34ICDFYGREZVM4AXZNU2ZYHWCA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:02.424981116 (83h5m0.976451587s) [noise=true , address=89.149.201.147:4014 , piece_id=2UFWXEUIRF5WQML3U54WFPMXLNPWZCZXC7SDSYQ7BW3WW6U6JMDA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:02.431720920 (83h5m0.983191381s) [noise=true , address=89.149.201.147:4008 , piece_id=36YUA6IMRY3NGBBUNUE7VTLJLKGFB2A64DKK5TDC35BKC2B57SFA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:02.489210783 (83h5m1.040681244s) [noise=true , address=89.149.201.147:4020 , piece_id=WH4KABC4AXFYVCL4ZMQO3OH4XMSJ2B5NGYUUK7475QO3ZFH25TCA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: <nil>
Nov 12 19:09:59.360142978 (83h5m57.911613429s) [noise=true , address=81.171.14.70:4043 , piece_id=2YAKQXKSM37TSETEE4HPZZJ3L343MCCPSLPA4HCLUA24TSAI4YWA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:41504->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:41504->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41504->81.171.14.70:4043: use of closed network connection
Nov 12 19:09:59.360485811 (83h5m57.911956272s) [noise=true , address=81.171.14.70:4029 , piece_id=YVZ525YBHPJSLQBT5VIB53WX5ILRVYJ2LIQHBFFXSG7NUWI2BUZQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: read tcp 188.166.225.170:39346->81.171.14.70:4029: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.360834505 (83h5m57.912304966s) [noise=true , address=81.171.14.70:4038 , piece_id=KUOMWLM4XZZAATAE7S6TGQH3OWEBUSWSTNRDEVKIBFN3YKZHMCTQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: expected piece hash; context deadline exceeded; EOF
Nov 12 19:09:59.360986263 (83h5m57.912456724s) [noise=true , address=81.171.14.70:4034 , piece_id=DK37C2K64H526QTS7D2LF7GUCWPLKWRDM4EXRWPXCCSH7G5IPKXQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN, address:81.171.14.70:4034): protocol: write tcp 188.166.225.170:57412->81.171.14.70:4034: use of closed network connection; write tcp 188.166.225.170:57412->81.171.14.70:4034: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57412->81.171.14.70:4034: use of closed network connection
Nov 12 19:09:59.361252441 (83h5m57.912722902s) [noise=true , address=81.171.14.70:4031 , piece_id=4EHA7SFBRDLVUIZVWYVYFMBIPFMQNUWXWHES3G67DA3UWDENQ4AA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.361391656 (83h5m57.912862117s) [noise=true , address=81.171.14.70:4028 , piece_id=PGXJBNZUEBOEAN2B3ZQ2WI7YXL7DX5EWZWXGTVCZPYRO6NEAFXRQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: write tcp 188.166.225.170:53368->81.171.14.70:4028: use of closed network connection; write tcp 188.166.225.170:53368->81.171.14.70:4028: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53368->81.171.14.70:4028: use of closed network connection
Nov 12 19:09:59.361475685 (83h5m57.912946136s) [noise=true , address=81.171.14.70:4046 , piece_id=ZTIK2PQVHVSXL2G2EMWYH6ZEKRNTMFSXJ5PVRVVQC2BRGTVWTHWQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: write tcp 188.166.225.170:51272->81.171.14.70:4046: use of closed network connection; write tcp 188.166.225.170:51272->81.171.14.70:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51272->81.171.14.70:4046: use of closed network connection
Nov 12 19:09:59.361557481 (83h5m57.913027942s) [noise=true , address=81.171.14.70:4032 , piece_id=2MS33WGPAP6EZA26XEG7TA52QT2LM6IVXRTZE34RGAZQDNGZOG7Q, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:47142->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:47142->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47142->81.171.14.70:4032: use of closed network connection
Nov 12 19:09:59.362211306 (83h5m57.913681767s) [noise=true , address=81.171.14.70:4025 , piece_id=WD4Z44PHRC555L4MGQ4MCHRNRRJJ45RH5XWX4DMIV7N4CM2GQQ7A, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:39230->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.362361833 (83h5m57.913832294s) [noise=true , address=81.171.14.70:4037 , piece_id=SIESKKRUKLABI6MFTPKB33I5GIZFQGY2D5CRJRXNL6OCIFBS4FXQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 188.166.225.170:33064->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.362505927 (83h5m57.913976389s) [noise=true , address=81.171.14.70:4036 , piece_id=7QE4NRQ2GUDGMOBO2YHU5XSBYPXCCNGC6ZXWW5X2VKBI3EZ7LLCQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:39160->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.362676262 (83h5m57.914146713s) [noise=true , address=81.171.14.70:4044 , piece_id=6ZVJ5R5KWRAVX3ZCTM7CXLQXELA7LWQ6UKAWBJKOP5V6SVRO7ZIA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: read tcp 188.166.225.170:50010->81.171.14.70:4044: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.362846206 (83h5m57.914316667s) [noise=true , address=81.171.14.70:4035 , piece_id=5E2MFPA3E7ILFCLQXIFLR5RD4RLPQIGSSKOZBSC7VJ6O34U2D2EQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 188.166.225.170:34266->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.363010267 (83h5m57.914480728s) [noise=true , address=81.171.14.70:4030 , piece_id=OX4DX73RI3RYGFLA7FBLS6IJC45WB6ZCL4IEG32KOSELDXURRHCQ, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:42334->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.363319596 (83h5m57.914790078s) [noise=true , address=81.171.14.70:4045 , piece_id=PHTSGMBHGXKACVMGAVNGPDKPZDRUDFQHDQ34WMZARQU3SU67RCKA, seg_pos={0 0}, upload=73135]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:52068->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 12 19:09:59.363403056 (83h5m57.914873507s) [seg_pos={0 0}, upload=73135]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil>
Nov 12 19:09:59.558309961 (83h5m58.109780422s) [upload=73135]: Done uploading segments...
Nov 12 19:09:59.558357222 (83h5m58.109827683s) [upload=73135]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13)
Duration: 1m0.199915076s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
3.385598437s trace:
https://jaeger.dev.storj.io/trace/473cad700880852e
FAIL (2025-11-12 16:29:51 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.385623633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
3.363227909s trace:
https://jaeger.dev.storj.io/trace/473cad700880852e
FAIL (2025-11-12 16:28:54 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.363262074s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
3.432629661s trace:
https://jaeger.dev.storj.io/trace/473cad700880852e
FAIL (2025-11-12 16:28:08 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.43266012s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
3.465935959s trace:
https://jaeger.dev.storj.io/trace/473cad700880852e
FAIL (2025-11-12 16:27:00 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.465968142s
core
uplink
us-select-1
us1-placement-20
|